First of all, the PS Minis and PS Classics (PS1 & PS2) that you buy from PSN will probably all work day 1 (or soon to it) since those are aren't actually PS3 games but are running on top of an emulation layer.
That being said though, Sony's only recourse to bring in PS3 emulation is going to have to be hardware based and I think I have insight on how it may be possible. Basically, all Sony would have to do is reconfigure an idea they already pioneered with their latest Vaio Z laptops.
That's an external GPU, Blu-Ray Drive, and USB 3 hub all combined into a single unit. You plug it in, and the already super powerful Vaio Z (most powerful laptop in the ultra portable 13in range, and is lighter and thinner than the Macbook air) becomes EVEN MORE powerful and gains all the benefits of a discrete GPU.
Sony could theoretically get a similar setup going on the PS4 but swap out the PC parts in the Vaio Z for PS3 parts instead. Sony already has patents on the idea when applied to Video game consoles so all I think they would need to do is get the PS3 components to a small enough size which will likely happen as they continue to sell the PS3 as a low-cost Media Center nexus that links into Sony's ecosystem.
If the PS4 has some kind of proprietary "accessory port" or any other ultra high-speed data port in the back, we'll at least know that something like this will be possible in the future and the Sony may be planning on re-introducing Backwards Compatibility.
Backwards compatibility could be marketed as a luxary feature and Sony could charge something like $150 - $250 extra. It seems like a fair compromise where people who REALLY want it can just pay for it, while everyone else gets funneled into the Gaikai stream.