I’m sure there’s some obscure key bind to go directly there

It’s just Cmd+Shift+H (for Home). The shortcuts for many of the most common locations are extremely intuitive.

• Cmd+Shift+A (Applications)
• Cmd+Shift+D (Desktop)
• Cmd+Shift+L (~/Library)
• Cmd+Shift+C (Computer)