Most likely, the answer to this question is yes. You can skip appealing to the Administrative Appeals Office (AAO) as long as there is no statute or regulation that explicitly requires an administrative appeal.
The majority of regulations governing USCIS petitions and applications do not require an AAO appeal. This standard was set (with more details, of course) in the Supreme Court case, Darby v. Cisneros, 509 US 137 (1993).
While skipping an AAO appeal carries some risk, it is usually a wiser strategy. AAO appeals are rarely successful and they give the Government an opportunity to bolster the reasons for the denial, which will make a federal court review more difficult. AAO appeals also take time and can delay the process of getting to an eventual approval.
If you had a petition denied by USCIS*, it is best to review your case with an experienced federal litigator to see if a lawsuit under the Administrative Procedure Act (APA) is a viable, cost-effective option.
* Naturalization denials are treated differently under the law and require a separate analysis.