Hold up, I just stumbled across the answer:
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
The viewport-fit directive is the crucial part. Once added, no more messing around with custom css, neither in landscape nor portrait mode.
Hope this helps others as well and finds its way into the docs.