In very near future your IDE will send the whole codebase as context to LLMs. Then instead of thinking up all possibilities you can just ask. LLM will suggest multiple alternatives and you can select the best when and ask it to implement it.
I said that the context window improved. I mean that it is larger. GPT-3.5 is 4k tokens, GPT-4 is 8k tokens (standard) or 32k tokens (only API access atm). This is the number of tokens that GPT-X can take into account when producing a response.
Specifically, I was using this to support the statement "In very near future your IDE will send the whole codebase as context to LLMs." I'm not talking about loops or accuracy.
It's true, but there is no indication that GPT can explain larger concepts for you, and negative indication it will be able to do it accurately.
It can't even explain small code to me unless it is something that it has been trained on. Often it gets even simple things wrong, either obviously, or worse, subtly wrong.
I agree that this is the part that needs more work, and is most uncertain. Increasing context windows seems like a fairly straightforward computational challenge (albeit potentially expensive). On the other hand, whether or not we can scale current models towards "true understanding" (or similar), is a total unknown atm.
I still think we will get useful things from scaling up current models though. I've already got a lot of value out of Copilot, for instance, and I'm looking forward to the next version based on GPT-4. Recently, I've been using the GPT-3 Copilot to write a lot of pandas/matplotlib code, which is fairly straightforward and repetitive, but as mainly a Java developer, I just don't have the APIs at my fingertips. Copilot helps a lot with this sort of thing.
GPT-4 has a model capable of using around 50 pages of written text worth of tokens (32,000) not sure exactly how many lines of code that translates to but it’s a lot. GPT-3 can use 4k so that’s a huge increase, the next version could be even larger and there are other ML techniques that allow for massive context lengths. Copilot already does a good job of refactoring code and knows enough about your code base to use your functions and methods. So what the other commenter said does not sound impossible to me.
It isn't anywhere near being able to diagnose anything more than off-by-one and other common errors.
Identifying hose problems will bring a LOT of value - but it isn't going to program and do general problem solving for you! It just has no signs of being able to do that.
Yesterday’s Steve Yegge post talks about it. You can provide text as context but you can also provide a dense representation in the form of text embeddings that capture the context. Today you can manually do it by something like LangChain but in future it will be part of our text editors.
Yes it will be able to give amazing feedback to us as devs and quickly identify common problems (that I still make all the time, even after developing for a decade!) which will bring a ton of value to programmers.
But, it will be a tool - it won't be something that will solve general problems for you. It won't make an average programmer a great programmer.
The moment that happens we can all forget about working and just do arts, space exploration, and acid orgies. But that “future” is somewhere between full self driving and thermonuclear reactor.
