A variety of situations exist in which an organization might find an accountant useful. Some use one only on a yearly basis, others more often. They can also be employed as paid staff, or possibly on a volunteer basis. Some accountants even specialize in non-profit organizations. It is important to make sure you're registered in states that require it (if you make >$100,000 for example).
One option is to have a part time bookkeeper who works with an offsite accountant <10 hours a week.
It is also quite possible to start with a volunteer or part time accountant and transition to a full time accountant, part time controller, with an outside firm coming in yearly.
An organization with a medium small annual could have a part time bookkeeper, a part time finance director, a consulting arrangement with an accountant, and an annual full blown audit, while the treasurer looks over the books monthly.
There are two yearly options depending on how large the organization is and what different states require. A CPA can do a yearly review, or a full blown audit yearly might be necessary.
Outsourcing accounting completely is an alternative for larger organizations, but a financial committee and annual audits usually come along with this.