A deck of cards is nice (water-proof or in a zip-lock.)
Those magnetic travel games (checkers, chess, backgammon, go, etc.) are pretty compact. With the random aspect of dice in backgammon I used to like to play against myself if no one was available.
Many of these games can be made with a little imagination & some basic materials. Perhaps a book of games/rules could be brought along instead & you just make the board/pieces.
If you are outside & have a knife, whittling would be a good distraction & even useful if you can make simple tools out of carved wood.

Tim