Armenia is perfect for adventurous backpackers. There is only one traditional hostel (the Envoy at $25/night) and a couple of hostelly homes for much less, right by the Opera, run by Anahit, so in Yerevan, you should be able to find what you're looking for. In other parts of the country, there are no hostels, and it's you, you're backpack, and a very friendly people who you just have to be open to communicating with in order to 1) get around on the minibuses and 2) figure out where you can sleep for cheap.
If you base part of your stay in Yerevan, you can see much of the stuff in Central Armenia pretty easily and cheaply by combining minibus (marshutni) trips and organized day trips with tour companies. Why? Marshutnis don't go everywhere tourists want to, only where there are towns and villages. So, for example, you can take a Marshutni to Echmiadzin, no problem, for a dollar, since there is a town there. Same with Ashtarak and Garni. But from Garni to Geghard there is nothing, so you either have to take a cab from Yerevan or Garni, or an organized day tour from Yerevan. If you get 4 backpackers together, taking your own cab can be worth it, but by yourself, a day tour with guide and all can be quite cost effective.
Because public transport schedules are not structured around tourists needs, you may sometimes face whether you want to end up in a small town or village without a set place to stay. Give it a shot! People are very friendly, and you'll have a place to stay if you ask around. You should certainly offer money and pay though, even a few dollars, since they'll no doubt feed you and might be sleeping on the sofa for you to be comfy. They will likely take offense to cash for their hospitality, so leaving some overtly in your bed, or telling them to take it for the kids is best. Alternatively, you can take them a small, useful gift, wine, and make sure to mail them some photos you take of them (and one of you and your family at home), which they'll definitely enjoy. The only caveat is that a lone female traveler should of course exercise extra caution - and tell people they're engaged/married/dating etc.