I'm sure there are many good choices but I'll share mine.
I have two Wilderness System kayaks which I really like, the Ride 115X Angler and the Tarpon 100. Both are very comfortable and very stable which was also one of my biggest concerns when buying. The Ride is the more stable of the two because it is a wider larger kayak. However it is heavier and precisely why I also added the Tarpon 100 and even though maybe not quite as stable as the Ride 115 it is still has more than enough stability that I have never been concerned and more than worth the trade off for the ease of use with the lighter weight and smaller size. I have a lot of back troubles and the Ride was a little harder on me when alone so I now mostly use it when I have some help. The Tarpon however I can easily carry by myself so it might be better suited for what you are describing you would want but if it weren't for my back trouble I could handle the Ride by myself more easily so you may like it better. With the Tarpon I can literally pick it up like a suitcase using the side handle. It is only 10' long so fits easily in a truck yet still large enough to be both stable and hold a lot of gear, I love it. I am not a small man and I usually take along a fair amount of gear and I can easily do so and have never felt unstable or in jeopardy of flipping it. I use it on anything from small to larger lakes mostly on Lake Barkley which can have some decent sized boats some which sometimes are going by me fast and fairly close or can also have some strong winds on occasions and I still feel very safe on it although I don't venture too very far out into the main lake with the kayak but I'm still in a fairly large area of water where I mostly go. Of course I would recommend that you always wear a PFD; especially so since you mention not being an experienced swimmer.
One thing I will mention is that I did have some prior experience paddling canoes when younger which I found helpful when I took up the kayak fishing and if you have never done much canoeing or kayaking you probably want to learn some basics because stable or not you still can flip them if you are careless. Not easy to do if you are careful so no reason not to enjoy the sport but you still should know the basics and use common sense.
You will love it though. There is something about being right on the water and paddling a craft while fishing that is unique unto itself. I have a 16' boat with an outboard and while of course it is fun and it certainly has it's own set of advantages there is nothing quite like being all alone paddling quietly on some water somewhere while fishing. I use my kayaks a lot.
Good luck with it, let us know if you get one!