OSM is definitely the rate limiting factor. I suppose that the more the site is used, the more caching will take place, right?
Not in a persistent way f you've created the OSM image server data source in the part of the project that's saved in the .map.
First, a few words on how caches work when you use them with image servers:
Suppose you're working in desktop mode on a project to which you have read/write access. In your project you've added an OSM image server data source, and you checked the 'Save cached data between sessions' option in the data source dialog.
The File - Create - New Data Source topic discusses in detail what that means and what capabilities it gives you. What it means is that tiles you pull in from the image server will be saved in a folder within the .map project file. Those saved tiles are the cache, and, like the other contents of the .map project they'll be saved in the project when you save it. When you open the project again those tiles will still be in that cache, ready to be used. That has two big benefits and two possibly significant downsides.
One big benefit is that your image server will seem to be running much faster, because any panning or zooming you do that requires tiles that have already been downloaded will happen instantly, since those tiles are already in the .map project and can be displayed much faster than even a fast Internet connection to OSM, Bing, or Google can get them. If you're working for a town and all your work is in that town, a side effect of the normal browsing you do in your projects will be to automatically load up the cache with most of the tiles for the image server you use as a background, so only rarely will 9 ever have to fetch some previously unused tiles from Bing or whatever.
A second big benefit is that if you open the project in a setting where you don't have Internet, say, on a laptop far out in the field, you'll still be able to use that image server layer just as if you were connected, so long as the panning and zooming you do falls within the tiles that were previously downloaded in your panning and zooming sessions. I use that effect a lot when I'm planning on doing some work in the field where I don't think I'll have an Internet connection (by phone or whatever) for my notebook. Before I go out into the field I'll create a Bing or Google satellite or streets image server with the cache option turned on, and then I'll pan and zoom in my map the regions I expect to be visiting, usually saving some locations so I know I'll have the key views I'll be needing. I save the project, and then when I'm out in the field I can go to those locations and I'll have the views I want from the image server even if there is no Internet connection.
But there are two main downsides. The first is that if you're doing a lot of panning and zooming with cache turned on, the cache can get really big, hundreds of MB, and thus a project that maybe had only a few MB of vector layers can end up being a few hundred MB in size because of the large size of the cache. It would be nice to have a "max cache" parameter set (I'll suggest that) to prevent projects from growing too huge if somebody doesn't notice how big they are gettting.
The second downside is that the system can't write cached tiles into the .map project if the .map project is read only. You can't create a new component in a read-only .map file, so you can't create a new table or add records to an existing table in a read-only project. If you opened the .map project read/write, created the image server with cache turned on and then used it, the project will have the cache file created. If you later open that .map project read only, tiles that have been downloaded during the current working session will be cached in virtual storage, but since the .map project file is not writeable when the session ends they can't be saved for future use.
It's that last point which affects how caches for image servers don't work past the immediate session with TCP or HTTP servers: even if you created the project with the cache option turned on for, say, an OSM data source in the project, as adamw mentioned, web data sources created in the MAP file will not be able to save the cache for future use because the MAP file is opened in read-only mode by the HTTP server. Server services (or command prompt sessions) are often stopped and started again to do things like make slight changes to the map being served, so the virtual cache will only be around until the server service/session is stopped.
How can you get around that? By creating the OSM data source in a read-write data source, like PostgreSQL. Create a .map project and link in a PostgreSQL server to which you have read/write access. Within that data source, create the OSM image server. The cache table will be created within the PostgreSQL server as well. You can copy the image from the OSM image server and paste it into the main part of the .map, and then you can use the image as a layer in whatever map you'll be serving with Server.
Save the file and close it. You can now use that .map file in Server. Even though the .map file's components (like the OSM image layer in the map) are all read-only, the cache is back in the read/write PostgreSQL database and can be updated.
Adamw gives the example of using an SQLite database. If you don't want to install and operate a PostgreSQL server you can just use an SQLite database file, linking that in and creating the OSM data source within that SQLite data source. That will work too. But my experience with SQLite is that it is slower than PostgreSQL, so I'd go with PostgreSQL.
this runs a little faster, but not too much.
That may or may not be improved by the other two important performance upgrades in in build 182.2, depending on what you are serving.
The first upgrade:
Rendering a map with image layers that are completely offscreen performs faster.
Big web maps often have very many layers, perhaps 20 or 30 or more, and some of those can be huge. The "Euro Dem" terrain relief layer that Manifold uses for testing which has high resolution terrain relief for much of Europe is around 190 GB. But that only covers Europe, so if you want to create a web-served world map that shows terrain relief you might use a background layer that's a lot lower resolution to cover the whole world with the Europe layer covering Europe. You could also have a hires, big terrain layer that covers the US, and for other regions of interest.
If people are looking at a spot in Asia in the map window, there's no need for the system to waste any time thinking about rendering an image layer that only covers Europe. Build 180.2 adds that optimization. Manifold's pretty fast so with a single user that's not something that's usually noticeable. For example, the "rendering 110 GB of images" video has some very large images in Boston, Paris, Hong Kong and so on but it still renders everything instantly without this capability, showing, for example, Paris very fast even though there are huge images off screen.
But when the HTTP server is handling possibly thousands of users, it makes sense to avoid any wasted effort. Even with one user if there are 30 or so layers that are huge and are covering different parts of the world, it can make a noticeable difference in speed to cut out any consideration of layers that are not in view. If you have such a map, 180.2 will be visibly faster.
There is also the second optimization:
The HTTP server caches rendered map images for performance.
You can see that in action by browsing the web map in your browser, and then clicking the Zoom in button to zoom, say, ten steps into the map. Now, click the Zoom out button ten steps. You'll see the zoom outs will happen instantly, because they're showing cached views. Click the Zoom in button and now those zoom ins will also happen instantly, because they're also now showing views that are in cache. Same also for the zoom to fit button, which now displays from cache.
This doesn't help if you don't repeat the same views, for example, by panning in random ways or by using the mouse wheel to zoom to different zooms than previously used, but it can still be a big help. As more navigation controls like locations and "back" and "forward" arrows are added, the cache will be a bigger help.
Hope that helps!