Yes, in winter (October until March) you can only drive there with a 4WD as it’s out of the main road and road administration services are clearing the snow not as frequently as on the main roads. Make sure to rent a 4×4 if you plan to get out of the capital. In summer, no 4wd is needed and the location is easily accessible also with a commond 2WD.
Please always check www.safetravel.is and road conditions on www.road.is before you go on a journey, they also have webcams where you can see the conditions of many roads.
You can find weather forecast and storm warnings on www.vedur.is