We've used the Crédit Agricole in Felletin for many years, and found them very correct. If you speak reasonable French, you can probably get them to call you at the number on the linked page, tho I don't know if they'll call you in the UK or not--you can try looking here, perhaps specifying you'd like to chat with an English-speaking agent. They do have an agence in Bourganeuf. Setting up an account would probably be easier if you went in person, but you could ask. Once you have an account, which in our case includes a checkbook (chéquier) and a couple of debit cards (one's charges accumulate and are deducted from one's account once a month), setting up automatic deductions for taxes, utilities, etc. is pretty straightforward. We're charged about 6 euros per month for the services of the bank. Of course, one can use the cards to get cash from ATMs (distributeurs de billets), of which there are many, everywhere, including most supermarkets. There are typically no additional charges for using ATMs, even those associated with other banks, anywhere in the euro zone. We do almost all of our financial stuff via their website, and I transfer money directly to them as needed, for which I use xe.com, whose rates seem competitive and whose service has been good.
I would strongly disrecommend the Banque Nationale de Paris (BNP)--we used them for some years, and they were awful.