Yes you can play with your PS4 without going online...
But can you save your game on your USB stick without the need of going online? Hum hum!!!
Now if Playstation is listening to custumers just let save on the USB stick lke a regular pc will do since a very long time ago.
This is my point of view and don't get me wrong here, I love Playstation
FYI
Made some research looks like this is the answer for PS3 games saves:
"It depends on the games not the PS3 itself. The system's fine with shuffling savegames around, but developers have created several countermeasures. Some games just plain don't let you copy the savegame and flag it correspondingly (Rock Band), many don't let you earn Trophies off a savegame you didn't make (Uncharted), others don't load savegames you haven't created"
Game-Heure currently has 4,471 views spread across 1 video for Haven. His channel published less than an hour of Haven content, making up less than 0.04% of the total overall content on Game-Heure's YouTube channel.