I decided to do some experimenting last night and try several different setups under different conditions to see how it affected factors like tracking and drift. I moved the camera around a bit and turned on and off different lights in the room, and tried to account for the possibility of light reflection. I was one of the people who didn't have too many issues with tracking, however I was dealing with a very aggressive case of drifting to the right.
I'm now mostly convinced that the drift problems (and the motion sickness I got) were highly dependent on the setup of the room, which was never really optimal in the 1st place. My couch sits relatively low to the floor in relation to the height of the entertainment center my TV was on, and I was concerned that the shiny leather (even though the couch is black) might have had an effect. All the recommendations I had read suggested the PS4 camera should sit atop the TV, but due to the height of the couch and television, this setup was really only optimal for, most likely, someone over 6 foot tall. I was having to angle the camera downwards and for whatever reason had to move the camera to the right to center it on the couch, which I tend to believe at this point was being caused by some sort of reflective factor.
After a few trials I found the best place for the camera was at the base of my television and draping a blanket over the couch seemed to mask the angling issues. Whether or not I had the soft room light on or not (a 20W warm bulb) didn't seem to affect things.
I played VR Worlds for about an hour and a half to two hours. Played a few rounds of Danger Ball, did an Ocean Dive, and finished the second half of Scavengers Odyssey, a game that had previously made me feel slightly ill. At first I was obsessively re-centering the screen like I had been doing when I was experiencing bad drift problems. But over time as I became more immersed in the environment I slowly stopped doing it, and during the final duration of gameplay that I mentioned above, I experienced almost zero drift and felt no motion sickness whatsoever.
Sure, the tracking can be a bit finicky, and that seems to be no matter what. A couch is not an ideal seat for VR, because it allows you to slouch back and your posture isn't controlled so you can easily get "lazy" and slouch out of position. I had to be sure I sat up straight as if I were in a chair with a backrest. Overall, I am now much more confident with this thing and I feel great about keeping it and not worrying about the drift.