Female led relationships discussion

Has your key holder ever forgotten you were locked? My wife and I have played with chastity for number of years, my longest lock up being 40 days! The only time my jailbird is removed, is if we go cycling, but on two occasions now, despite my key holder knowing that cycling whilst wearing is extremly uncomfortable, last time as we prepared to go for a cycle ride, I had to ask for the key and my wife thought I meant the car or the door key! I am right to be upset or should I just get a life and get on with it?