France is beautiful, I had several 'gipsy' vacations there.
Going by train is probably not the best option if you want to visit the countryside, you may want to evaluate the mixed option to set as milestones 2-3 big cities to reach by train and then rent a small car for wandering in the countryside.
You can then drop the car at the next train station and continue to the next milestone by train.
As areas that I would recommend, I say Normandy, Bretagne, Beaujolais, Camargue (ate wonderful fish) and the Alps regions. Definitely Provence if you go in springtime.
I was not greatly impressed by the Cote d'Azur.
For attractive places, you can rely on the French: they are usually very good in advertising places of interest with signs along the road.
I always find great accomodations with this guys:
Logis de France : Leisure escapades and Logis for hiking or other sport activities
It is an association of small family-run pensions normally with a more than decent restaurant attached (although is very easy find a good cheap restaurant or bistrot in any small French village)
I would suggest to buy their book and decide day by day where to stop for the night.
It worked very well for me: I used to book the night during lunch, and plan the next day route after dinner.
The best for me was by motorbike, always driving quietly in the country roads, spotting a nice place and just go there: i have found many unexpected little jewels, like antique villages, church, towers etc.
have fun!