we got style

It may sound stupid, but I had to think about it. Well simply do:

apt-get autoremove mod-pagespeed-beta && /etc/init.d/apache2 restart

and mod_pagespeed will be gone!

Update: remove the cache, too:

rm -fr /var/mod_pagespeed/