Well i checked the files when i create the Hyperlinks within Excel. They are identical for the link. Then i checked the file created by ClosedXML and the XL2010.
Here is the difference:
So it seems that XL2010 can handle the incomplete URI and changes it internally to the correct format. The next save will fix the issue and XL2007 is also able to open the file.
I changed my coding to get it fixed to:
Dim uri As New Uri("C:\Users\Me\Downloads\Whatever\File")
ws.Cell(1, 1).Hyperlink = New XLHyperlink(uri.AbsoluteUri)