I’d bet good money that I was simply testing myself wrong.

How well the rapid tests work depend on a few factors, but the ones I want to focus on are (a) how wet your epithelium is when you swab it and (b) how many of the virions are hanging around the section of your epithelium you swab. If you don't breathe through your nose very much (easy to do when you have a respiratory infection), you might have fewer virions in your sinuses (depending on if you have viral replication there). If your epithelium is super dry (super easy for this to occur when its infected with a virus that reduces circulation, makes the throat sore (leading to less water intake), and makes a person cough a lot), you'll transfer a lot less viral matter onto the testing swab.

There are probably videos online that can help, but if your tissue is super dry the threshold of number of virions produced by the subject per second required to register a positive probably increases.

Source: speculation mostly. I used to do research in a different but related subfield of biophysics