Settings reputed to be too weird to effectively run are about as old as RPGs themselves; Empire of the Petal Throne was, when it originally came out, reputed to be unplayably dense and weird, though I suspect a lot of that stems from it taking more inspiration from South Asian and Mesoamerican cultures than it did from European history. In the 1980s, the crown of “wackiest setting” belonged to Skyrealms of Jorune, famed for running a series of adverts in Dragon magazine like this one:
And to be honest, right there you can see both what people found appealing about Jorune and what people found off-putting. The artwork is impressively weird, and combined with the dialogue it takes on a whimsical, almost Vancian quality. At the same time, the actual dialogue is full of fantasy gibberish words and suggests a fussy, overprecise setting with lots of fiddly details.