I've played around with it a bit, and had it spit out a few story-like things to see what it was capable of. It's certainly impressive.
As far as actually using it to write a story, I feel like that would take the fun out of writing for me. In addition to plotting & worldbuilding, I enjoy the sentence-smithing part of writing, and I don't think GPT is at a point where you can exert enough control to actually get the story you wanted to write. I'm sure you could write a full story/storygame with GPT, but it would be at the cost of the author's creative control. Character/narrator voice is such an important part of a story, and it's hard to see how you could get a strong and distinctive character voice out of GPT as it currently exists.
I expect that eventually writing aids like this will be about as common as calculators. Maybe less so in the artistic/creative sphere, because many people like sentence-building, and even with a far more sophisticated tool, you're still going to be sacrificing a lot of creative control in every area. But I do expect people are going to be using these tools to write publishable novels in the near future, if they aren't already.