Random capitalization of text (Mac OS jumping into Windows notebook)

I have an intermittent issue where I am typing text on the Windows remote computer (which is in the same room as me) and my text will capitalize, even when caps lock is not active on both the host or remote computer. Although I can decide to put the Caps lock (ON) to get lower case letters again, if I hit a full-stop it presents the '>' character instead of '.'

The issue seems to come and go on its own, but has anyone else experienced this? When it does occur, it disrupts my workflow. 

The Windows laptop is on Windows 10. I am currently on Mac OS Sonoma, but I've had the same problem for months on the previous version.