I was looking into this kind of lacked some details. Here is also another blog post that was quite helpful.
Sweepers require a controller, so cannot be used outside of a request. If a sweeper is called outside of the controller, all calls to expire_ will be silently ignored. You do not have to add a sweeper to observers, this is on purpose as the cache_sweeper method used in the controller adds it to the observers and also includes the controller object.