In Yerevan, your hotel and the tourist information office will be invaluable sources of information and guidance. If you are staying more than a couple of nights, renting an apartment might be more comfortable and cheaper than a hotel.
Most of the nicer restaurants and places to eat will have no problem giving you a receipt in Yerevan, but outside of Yerevan it is a pretty uncommon request, and you shouldn't be surprised to see it handwritten on a piece of paper. You can ask them to stamp (pechat) it if that helps make it more official. If you are staying at a hotel or B&B outside of Yerevan, be sure to inform them when you negotiate the price that you need a receipt! The price may change if you ask for one later, and in any case it can sometimes take them half an hour or an hour to create a receipt if you ask for it at 7 or 8 in the morning, so ask them to prepare one the night before.
When it comes to taxis, if you need a receipt, use a taxi service, with meters, and there will be no problem.