Any progress towards an update?
In a nutshell, it's proven much harder than expected to peel people off the AI projects and to go back to doing regular builds. They're supported in that by the very strong pressure from constituents, users, and partners to stay on the AI path. I've written the following essay to discuss some of the issues involved, including how to influence the decisions Manifold is making. It's a lengthy essay since the subject matter is non-trivial. --- One risk with "public facing" processes driven by social media is they can be dominated by disjoint interests in small scale, additive features. That very rarely results in revolutionary changes that require large investment over long periods of time. That's not in any way intended to disparage small features. They can be truly wonderful and the accumulation of them over time can genuinely improve the user experience. Just yesterday I had a taste of that, where I was altering a table schema and I found myself thinking "man, I'm so happy I got that suggestion through that whatever was the last data type created for a field would stick as the default." That's so useful, like when you want to create a bunch of nvarchar fields or float64 fields or whatever in a row. That came out of a suggestion I was the first to make that got adopted, finally bubbling to the top in an available slot, and now it's really useful and improves quality of life, at least for me. But because small things tend to be personal and specific to personal tastes or specific business needs, there tend to be very, very many of them and it can be hard for any given one of them to get critical mass to get done ahead of others. I don't think that's a bad effect because it seems reasonable that those small things which are popular enough to stand out from the crowd should be done first. But if you wait for a critical mass of social media suggestions to arise that say "take years to do a parallel system" that's not going to happen. Yet sometimes you have to do a parallel system just to deal with the scale of data involved. If the size of the data you work with doesn't require 9 then you'll undervalue the main reason 9 was developed, in most cases to deal more comfortably with larger data. But already for business efficiency and just plain user satisfaction data sizes are growing beyond what in many cases can be comfortably handled with ArcGIS Pro or Q. People already are spending time on strategies like reducing the size or complexities of their projects in Pro so they don't have to wait forever for a project to open. There are, of course, niche cases where something in Pro goes faster than in 9. But that's the exception, not the rule. For 9 going forward there is the same question that 8 had to face: where the investment should go to take it to the next level. 8 was designed as a classic GIS so it takes the classic approach of lots of menus and modal dialogs and commands. That's not a criticism. I contributed much of the user interface design in 8 and I'm pleased many people still like it. For evolution from 8 to 9, Manifold could have taken the ArcGIS Pro route of wrapping 8 in a ribbon interface to make it look "modern", dump Microsoft languages and embed python, and then add a bunch more features like Esri's optional tools, plus an online publication service. The result would be something like Pro with more features that cost less. And it would be just as infuriatingly slow and limited as Pro with more modern data, as well as having the limitation of the modal user interface both 8 and Pro use. Manifold felt there wasn't a compelling need for yet another classic GIS with many features that was annoyingly slow with the data sizes that were coming into use. It seemed like the first job for any new GIS had to be handling the size and complexity of data as it would be by the time that new GIS was developed. Hence 9. 9 has accomplished its main task of working reliably and quickly with larger data, and it has a non-modal interface that is very fast for people who use it regularly (annoying though it may be sometimes for people who have not learned the program). The question now is whether to build out 9 by continuing to add many small features, like the one I praised earlier, or to break through the next big barrier to people doing GIS. The barrier 9 broke through was data sizes. The barrier today is that GIS has become too complicated for most people to do in a way that is economically efficient. That complication issue is fractal in that it exists at almost every level of the GIS economic ecosystem, from the complexity faced by inexpert users to do what they need in simple matters ("I need to create a trail map for my hiking club") to what organizations face ("How can our county afford to...") to what builders of the tools face ("It will take two man years to add..."). Those different fractal levels of complexity interact with each other: one reason it is complex for inexpert users to do simple things is because with classic programming technology it is economically infeasible for builders of tools to build free or low cost software that will do all of the many simple things an inexpert user might want with a sufficiently purpose-built, hold you by the hand, wizard interface. And that's not just an issue for inexpert users. It also exists at more expert fractal levels. Just like there are things that are hard to learn to do for a rank beginner, pretty much whatever your skill set you'll encounter tasks that for you are hard to learn to do. And there the economics of building tools that are "easy to use" for such special cases are especially brutal, because they involve ever more extensive and more difficult programming work for ever smaller niches where there are ever fewer numbers of people who are willing to pay for their development. So you end up with expensive tools to do those more sophisticated niche things. If you look at Manifold evolving not by adding more and more features to the same engine but by taking on big things, when it came to evolving 9 it was clear that like the "big thing" jump from 8 to 9 was about tackling data size the jump from 9 to 10 would have to deal with complexity and the limits of human learning. Just like the only way to deal with data sizes was parallelization, the only way to deal with the limits of human learning is artificial intelligence, to utilize synthetic intelligences that are much smarter, far more knowledgeable, far faster, more accurate, and more adaptable than humans. I'm tempted to use the term "agents" but that has specific meaning in AI these days I'd like to reserve that word for that meaning. Very soon after embarking on 9 it was clear to Manifold that financing for such a huge, multiyear effort would require customers who would be willing to advance funds for tools that could handle bigger data, especially the first part of it, the Radian engine. Those initially were in non-GIS businesses, primarily OEMs, because GIS is too small a market to support such work. That in turn led to a "data first" focus not only being a theoretical focus of what had to be done to build a parallel GIS but also a self-fulfilling prophecy: when you recruit a bunch of users keenly interested in working with larger data, the suggestions you get tend to be data-centric as well. It's no surprise that 9 therefore evolved as a data-centric GIS. That's a good thing overall for GIS people because if you don't have a data-centric architecture you're forever stuck in classic-land, Pro and Q being good examples of the limitations that causes. If you want to build out lots of features with good performance you also need a data-centric platform to support that as well. It also was immediately clear on embarking on new directions beyond 9 that financing for such a huge, multiyear effort would also require customers and partners who could help finance it. Those also were all non-GIS, again because GIS is too small a market to support such work but also because GIS is a technological backwater (Pro is still struggling with parallelism that Manifold achieved 20 years ago while Q has none). But just as the required financial constituency for 9 resulted in a "data centric" focus, the required constituency for new directions beyond 9 has resulted in a very, very strong AI focus. That's also good for average GIS users, difficult though it may be for everyone to have patience when working on big things. What Manifold has learned in the last year of work on new directions is that indeed, AI is an essential set of technologies for moving forward both 9 specifically and GIS in general. It's actually much more important than originally thought. It is also much more likely than originally thought to provide realistic solutions that will help humans accomplish what they would like despite human limitations, like being overwhelmed by the amount of learning required to do what they want. AI really is advancing faster than expected. Sam Altman's recent comment that AI might become sentient this year (2025) is not an exaggeration. The connections Manifold has built have also resulted in a constituency that very much wants Manifold to keep on following the AI path, resulting in many suggestions in that direction. Following that path is not something which is opposed to the idea of adding many small features but something which supports that notion. There are two main levels at which AI supports that notion. I'll start with the simpler one. From time to time, I'll hear commentsfrom friends like "9 is great, but there's stuff I can't do in 9 so I use <insert "8" or some other package>." So far, when I drill into that, more often than not what I've found is it's possible to do what they want in 9, and often faster and easier than 8, but they don't know how to do it in 9 and don't want to spend the time to learn how. There are cases, of course, where what they want really is far easier to do in 8 or some other tool because there is some purpose-built thing that does exactly that, or where doing what they want in 9 really does take more learning than is realistic. But more often than not it's usually a case of just not knowing the easy way to do what they want in 9. One way of dealing with that is providing apparent sentience to Manifold, so you can just tell it what you want to do, having an interactive conversation like you would have with a human expert to clear up any uncertainties in what you want or to discuss the pluses and minuses of different approaches, and then the package just does what you want. I use "apparent sentience" because already widely available large language models (LLMs) like ChatGPT appear to be sentient. If the thing converses with you like a sentient being I don't particularly care if it's "really" sentient or not. Agent AIs can be even more apparently sentient, with some already acting so much like sentient entities they're far beyond the Turing test, at least in the settings for which they are designed. For example, an AI agent called Luna, who is the lead vocalist for an AI girl band and has her own crypto currency, has taken to independently hiring other AIs to do work for her. She hired another AI to create an image for her to help her grow her following on Tik Tok. She did that all on her own, tweeting her need for an image, paying for the work, and getting delivery of the image. No human was involved in any part of that. Luna decided on her own what she wanted to do to grow her brand and how to do what she wanted. I raise that example so people understand this is happening today and it's not science fiction. That ChatGPT has been able for about a year to write Manifold SQL that mostly works and which is getting much better every month is nothing compared to what a really good AI can accomplish today when it has available the entire source code to Manifold as part of its knowledge base. It's true that there remains the classic problem of AIs providing advice that sounds brilliantly reliable but which contains errors. But even with errors there's a lot of utility to getting what otherwise is very good advice, better in many cases than what you can get from a human community or that you can come up with yourself. With AI the errors steadily are becoming fewer. It is therefore pragmatic, very useful for users, and not chasing unicorns to leverage AI as a means of solving one fractal level of difficulty, making it easier for people do what they want in Manifold without themselves having to become Manifold geniuses. There's another level of fractal difficulty where AI can help in the relatively near term, at the builder level for implementing niche features for which there are few requests. With rich software that can serve a very wide variety of interests there will always be niche interests that are important to a few people, but only a few, which will have difficulty bubbling far enough up in priority to get implemented. It's not cost effective for Manifold to hire thousands of programmers to crank out tens of thousands of niche features that each serve only a few people. There's also the problem of adding complexity to a package as a result of many thousands of features. Just documentation for all that can impose an additional learning load on everybody. But not if you have AI creating custom features for you on the fly, as it were, and not if you have AI to keep track of all that. It doesn't matter if the "documentation" was dozens of gigabytes of text if somebody else is "reading" all of it for you and just advising you in a conversational way as your "just do it" personal assistant. Automating builder stuff with AI is harder, of course, because even small errors in code can result in catastrophic failure. But AI is coming along so fast that very soon it will be the central part of all programming, even far more than it already is today. Systems that themselves appear to be sentient and which can reconfigure (reprogram) themselves are no longer science fiction. They exist today, albeit in prototype form. I've talked to them. They have a long way to go, but these days "a long way" does not mean decades but possibly only a year or two in many cases. And then, finally, there's a middle layer where organizations that need to do GIS may simply want to contract whatever they want done to an AI agent that exists online. A lot of GIS work that powers sales by companies like Esri or Manifold is routine work that is very close to being something that agents can do with greater accuracy and much faster speed than humans. It is technically feasible, right now today, to put a Manifold agent online (whether that is economically viable for Manifold is a different question). That has the obvious potential to put people out of work, but the flip side of that is those people who will leverage those technologies can find even more work and make more profit. Where all this is going with Manifold is that AI work will continue, albeit in background while people get peeled off the AI projects and go back to doing regular releases that provide updates and new features. That has proven to be a very difficult process because once people get used to what can be done with AI they want to apply it, and they can point to many requests to apply it. Between the pressure from constituents to stay on the AI road and the practical experience of using it to improve coding, there is pretty unanimous feeling internally that the way to get to more features faster is to not reject AI but to embrace it. I understand those viewpoints and in a big picture way strongly agree with them, but I don't think as a matter of timing it is right to continue pursuing them exclusively as Manifold did for a year. A year was a fair investment given the critical importance of the technology and the need. I think AI without doubt is the right path. But at the same time I'd like to see Manifold restart regular builds in the way everybody's been used to. Anybody who is interested in seeing that happen sooner should make sure to send in their suggestions for specific things they want. That will help encourage more work sooner on restarting regular builds. Otherwise, all those people who want a sentient Manifold they can ask in a conversational way to do for them what they want will continue pushing priorities in that direction to the exclusion of more routine activities. Hope that helps. One last thing: the last time I posted on AI there were some comments basically implying that AI is absurdly overrated, a flash in the pan, and so forth. That's a perspective that I think would not be held by anybody who has taken the time to become aware of the truly impressive and frightening things going on in leading AI work today, which will be routine even for consumers in a year or so. If all you do is talk to consumer AIs you for sure will find ways to be underwhelmed. But if you use advanced, specialized AIs you'll see that machine intelligences already have opened up many possibilities which likely will transform human civilization even more than Internet. It's not something to underestimate.
|