Prevent side-by-side inputs always focusing on first input

Hi, I’m having trouble working with side-by-side inputs. Basically I have 4 inputs which I have nested within the same <label> element, as I want them to show side-by-side. I have managed to achieve the look I want, however whenever I click on the label row or on any of the inputs within the label, the very first Input box is automatically focused on / selected. This is obviously problematic if you want to enter anything into the second Input box, as the focus will move to the first Input box upon click. Is there anything I can do to remove the autofocus?

Please check out my Code Pen.