I don't know the way to best do that (otherwise I'd be talking at a board meeting instead of here) but I don't think it's just Vita+better hardware.
You would be correct. Vita+better hardware is not the way to go. Oh, better hardware would be nice. But it shouldn't be the point. It doesn't need to be the point.
And you're on the right track with the on-line PSN store, but I suspect that the use-case you are describing is for more of a niche audience. Not everyone is interested in inexpensive digital imports. (though I can certainly see the appeal) Discussing a global on-line store would be something Sony would have to collaborate on with their development partners. You can't just force that sort of thing, though it would be possible to offer it as an option to developers.
Take a step back, analyze the industry, and get a feel for what the competition is doing. Then make some informed predictions about the direction things are going. That is how you plan for a new product. With the PS4, Sony re-focused their efforts on games. They stepped back from the media machine they had been trying to build with the PS3, and instead focused on what had made their gaming division successful in the first place. A tight focus on games, and really solid third-party development support. All of the design decisions for the PS4 are focused on those strengths.
Now look at the current portable market. You've got mobile platforms, which are hugely successful, but have grossly underpriced mass-market software. Most of their stores are just straight-up stuffed with shovel-ware, and discoverability is a real problem. Then you have dedicated platforms like the 3DS and Switch. The Switch is competition to worry about, Nintendo did a lot right with that system. The 3DS is dangerous because its so well established, and can be very competitively priced. Those are some substantial obstacles, and they need to be taken into account. They have weaknesses to exploit as well, though.
Now take a look at Sony, and figure out what their core competencies are, and what advantages they can leverage. Sony has solid industrial design. The PSP, Vita, and Vita slim all had smooth, appealing exterior design. They can make a good-looking system that will appear slick and appealing on-the-go. They have decent developer relations and support. These qualities have been re-inforced with the release of the PS4. Now is the best time for them to leverage that strength. And perhaps most significant, they have a strong on-line infrastructure and store, as well as a sizable heaping of digital offerings. This is an advantage that few, if any, of their competitors can boast of. Nintendo has always been behind when it comes to on-line support, Sony could leap-frog them easily on this front. And mobile storefronts have become choked with un-regulated run-off. PSN's more closely managed storefront is vastly more appealing, and considerably less clogged with gunk.
My personal suggestion is a mobile Sony platform with mid-range hardware, specifically geared towards portability. If they make it more powerful than the Switch, they've probably gone too far. It would be ideal if they could squeeze a mobile-optimized x86 chip into it. Many of the major chip manufacturers have begun optimizing their CPUs for laptops, resulting in CPUs with much lower power draw that can run cooler. Normalizing the architecture across their mobile and console platforms would make porting to the mobile platform a complete breeze, which would make developers very happy, even if they have to sacrifice power to make it happen.
The really big question is the controls. What do you do about those? Nintendo kind of knocked it out of the park with the Switch's Joycons. But I'm not sure it would be best to ape that design. Allowing people to pair their own controller options via blue-tooth is a given. You should be able to pair the portable with a DualShock 4, right out of the box. A dock would also be a good idea, but I'm wondering if they should make it as a secondary add-on. I'm also thinking that you should be able to plug the portable into a PS4, and essentially use the PS4 as a dock if you want to. (pass the video from the portable through the PS4, not sure if this would be possible with USB 3)
And of course, any and all legacy digital PSN offerings should run on this thing. PS1 games should run on it. PSP games should run on it, and digital PS Vita games should run on it.