Exhibit 5.1 

LUSE GORMAN, PC

ATTORNEYS AT LAW

5335 WISCONSIN AVENUE, N.W., SUITE 780

WASHINGTON, D.C. 20015

TELEPHONE (202) 274-2000

FACSIMILE (202) 362-2902

www.luselaw.com

June 4, 2025

Board of Directors

Orange County Bancorp, Inc.

212 Dolson Avenue

Middletown, New York 10940

Re:      Registration Statement on Form S-3

Ladies and Gentlemen:

We are acting as special counsel to Orange County Bancorp, Inc., a Delaware corporation (the “Company”), in connection with its Registration Statement on Form S-3 (File No. 333-280793) (the “Registration Statement”), filed with the Securities and Exchange Commission (the “SEC”) under the Securities Act of 1933, as amended (the “Securities Act”), relating to the proposed public offering of 1,978,494 shares of common stock, par value $0.25 per share, of the Company, which includes 258,064 shares subject to the underwriters’ over-allotment option (collectively, the “Shares”), pursuant to the Underwriting Agreement, dated June 3, 2025, by and among the Company, Orange Bank & Trust Company (the “Bank”) and Piper Sandler & Co., acting as representative of the underwriters named therein (the “Underwriting Agreement”). This opinion letter is furnished to you at your request to enable the Company to fulfill the requirements of Item 16 to Form S-3 and Item 601(b)(5) of Regulation S-K, 17 C.F.R. § 229.601(b)(5), in connection with the Registration Statement.

In rendering this opinion letter, we have reviewed copies of: (1) the Underwriting Agreement; (2) the Registration Statement; (3) the base prospectus, dated July 19, 2024, which forms a part of the Registration Statement; (4) the preliminary prospectus supplement, dated June 3, 2025, in the form filed with the SEC pursuant to Rule 424(b) of the Securities Act; (5) the final prospectus supplement, dated June 3, 2025, in the form filed with the SEC pursuant to Rule 424(b) of the Securities Act; (6) the Certificate of Incorporation of the Company, as amended; (7) the Amended and Restated Bylaws of the Company; and (8) certain resolutions of the Board of Directors of the Company and committee thereof. We have also reviewed such other documents and made such other investigations as we have deemed appropriate.

In our examination, we have assumed: (1) the genuineness of all signatures; (2) the legal capacity of all natural persons; (3) the authenticity of all documents submitted to us as original documents; (4) the conformity to original documents of all documents submitted to us as copies thereof; (5) the Underwriting Agreement has been duly authorized and validly executed and delivered by the parties thereto (other than the Company and the Bank); and (6)  the Shares will be issued and sold in compliance with applicable U.S. federal and state securities laws and in the manner stated in the Underwriting Agreement and the Registration Statement. As to various questions of fact material to this opinion, we have relied upon the statements contained in the Registration Statement and statements of officers of the Company, and we have made no independent investigation with regard thereto.

Board of Directors

Orange County Bancorp, Inc.

June 4, 2025

Page 2

Our opinion expressed herein is limited to Delaware General Corporation Law and we express no opinion with respect to any other laws, or with respect to any matter pertaining to the contents of the Registration Statement, other than as expressly stated herein.

Based on the foregoing and subject to the limitations, qualifications, exceptions and assumptions set forth herein, we are of the opinion that the Shares, when issued and sold against payment therefor as provided in the Underwriting Agreement, will be validly issued, fully paid and nonassessable.

This opinion has been prepared in connection with the Registration Statement. We assume no obligation to advise you of any changes in the foregoing subsequent to the date hereof.

We hereby consent to our firm being referenced under the caption “Legal Opinion” in the base prospectus and under the caption “Legal Matters” in the prospectus supplement and for the inclusion of this opinion as an exhibit to the Registration Statement. In giving such consent, we do not hereby admit that we are in the category of persons whose consent is required under Section 7 of the Securities Act of 1933.