Unfortunately I don't have any personal experience but a friend of mine went to Mongolia for a month's riding holiday last year and she said it was a brilliant experience. They did go with a guide but it was a small, experienced group and it sounds as though they had plenty of freedom e.g the guide didn't mind them trailing a long way behind if they wanted, he was merely there to show them the way to their stops each night. Mongolia is such a big country with such a tiny population for it's size that I think that it'd be hard to find your way in the vast landscapes with only a map and without anyone with local knowledge.
They stayed with families in traditional "gers" each night which my friend said was an incredible experience, often they were tiny settlements in the middle of nowhere! And she said that the people were extermely friendly and hospitable.
I think Mongolia would be a briliant place to go riding for a month. I think the vastness of it's landscape would mean that you would feel really free and at one with nature, which as you would like to go without a guide, I think is the kind of experience that you want. However I think you should go with a guide, it would be too risky to go alone as a couple into the wilderness. If something went wrong, one of you got ill or injured, it could be difficult to get help if you had travelled a long way from civilastion. At least if you went with a guide you would be sure of having somewhere to sleep each night, it's got to be safer than camping by yourselves.
My friend's very adventurous and a good rider and she was very happy after the trip so I'm sure the guide wasn't too restrictive with them. They did a lot of galloping!
Frequent flyer, i know that you specified that you would like to go without a guide and so I'm hoping that DavidS8 will respond to Emil's request for more information about the destinations he suggested. I just wanted to let you know of my friend's (wish it was mine!!) experience, maybe Mongolia might be somewhere to consider.
I've read a bit about Mongolia and it's really interesting culturally and beautiful from a scenery perspective. Horses are embedded in the Mongolian culture, in fact Mongolian Ghengis Khan and his 'Devil's Horsemen' created the largest empire the world has ever seen! It is populated by over 15 distinct ethnic groups. Mongolia is geographically diverse, with wild natural beauty in the form of forests, desert, steppe and lakelands.
Wherever you chose to go I hope you have an amazing time and share your experiences with us, would be great if you put your pictures up on this website!