Jamieson, Fausset and Brown on Genesis 40:4

4. The captain of the guard charged Joseph with them--not the keeper, though he was most favorably disposed; but Potiphar himself, who, it would seem, was by this time satisfied of the perfect innocence of the young Hebrew; though, probably, to prevent the exposure of his family, he deemed it prudent to detain him in confinement (see Psa 37:5).

They continued a season in ward--literally, "days," how long, is uncertain; but as they were called to account on the king's birthday, it has been supposed that their offense had been committed on the preceding anniversary [Calvin].