Well you have to answer yourself what exactly is scary about “infinite”. For me, its the visualization of something reaching a threshhold of “zero visibility” or unreachability, with the furthest point being a black dot on the horizon. Its about unknowledge.
For reference I think House of Leaves is an excellent example of infinite geometry. It exceeds pastas like NoEnd House; theres no gore, no monsters(?), just a very cold, dark labyrinth and a giant mystery.