I managed to figure it out.
It only works when using TCX files (not when using GPX files).
RWGPS allows to add these cues (in the paid version).
But you can also add them manually in a TCX file:
<CoursePoint>
<Name>Water</Name>
<Time>2023-10-19T17:13:09Z</Time>
<Position>
<LatitudeDegrees>x.xxxx</LatitudeDegrees>
<LongitudeDegrees>y.yyyy</LongitudeDegrees>
</Position>
<PointType>Water</PointType>
<Notes>Water!</Notes>
</CoursePoint>
This will give a warning with the text ‘Water!’ when approaching the point, and will show a water tap icon. Other type examples are “Food”, “Danger”