In the process of developing a single-page-application (SPA) with ASP.NET MVC, knouckout, and various other libraries, we have decided to handle routing on the front-end, potentially utilizing crossroads.js. Our use of slickgrid.js allows us to present a wealth of information in virtual grids, all of which is fetched from the backend using AJAX.
If the goal is to crawl and index such a site from SharePoint, the process becomes more complex. When attempting to load the main page without javascript, it appears almost empty.
Update
Further investigation into this issue has led us to consider two potential solutions.
Possible solution 1: Render HTML
An approach involving the detection of a SP crawler and the return of static HTML pages for the crawler could be implemented. Utilizing tools such as PhantomJS may be necessary, although there are uncertainties and potential challenges associated with this solution.
Possible solution 2: Import data into SharePoint
As explained by Josh below, importing the data into SharePoint for crawling eliminates the javascript dependency. By configuring SharePoint to crawl this data, it simplifies the process. This option seems to be the most efficient and straightforward, which is why we have marked Josh's response as the accepted answer.