Balleen is literally inside a whale's mouth. Hell, your teeth are non-living extruded structures, but they're obviously still part of the body.

Like you say, sponges and sedentary molluscs actively pump water through themselves. I would not class most jellyfish as filter feeders, but as passive hunters, which is again a separate category from either of these.

Here's another way to think of it: filter feeders use a structure to parse through their given medium ([almost?] always water), either actively moving through the medium, moving that medium over/through their filter mechanism , or both. The primary thing they are interacting with is the medium itself. Spiders and ant lions and other analogous creatures are altering their surroundings to be conducive to hunting. They are essentially little environmental engineers.