In short, yes. You must also begin and end your trip in the same country, without making landfall on the opposite shore.
If you are American citizens, beginning and ending your trip in the US, follow the instructions on the USBP website, here. We suggest that you send a trip plan to the nearest US Customs office in advance of your trip, to speed things along. So long as you do not visit a "foreign port," make landfall in Canada, or meet other boats mid-stream, you'll be fine. You can also cross over to Canada and make your trip entirely on the Canadian side.
If you are Canadian citizens, you can begin and end your trip on Canadian soil or US soil. Before your trip, report to Fosterville, St. Stephen or St. Andrews wharf, or call the Marine Telephone Reporting Centre at 1-888-226-7277. They are going to ask you questions, so be prepared. Visit CBSA's webpage on reporting so you have the information you need.
If you are a frequent boater, we strongly suggest applying for the appropriate program (I-68 or CANPASS) to speed up the process.
We also suggest that you contact the appropriate border agency if you have any questions or concerns about your trip at all. We want you to enjoy the St. Croix, safely and legally.