Thoughts, Trends, and Questions
The cost of inference is trending towards zeroToken throughput is trending towards infinityContext windows sizes are getting largerCompanies are spending more on training despite improvements in compute and cost efficiencyModels are quickly becoming commoditizedCompute is quickly becoming commoditizedWe’re sharing our notes on trends that we wrote about back in December of last year (and updated in February of 2024). This document has been sitting in our team Notion workspace for almost half ...
Thoughts, Trends, and Questions
The cost of inference is trending towards zeroToken throughput is trending towards infinityContext windows sizes are getting largerCompanies are spending more on training despite improvements in compute and cost efficiencyModels are quickly becoming commoditizedCompute is quickly becoming commoditizedWe’re sharing our notes on trends that we wrote about back in December of last year (and updated in February of 2024). This document has been sitting in our team Notion workspace for almost half ...
What We're Working On
Just two years ago AI could only retain about 8 pages of informationNow, it can memorize the equivalent of 10 King James BiblesContext window size determines how much models can remember, and it is growing at a staggering rate—1,250x each yearEverything you do, see, or hear now fits as memories in an AI’s context windowCombined with increasingly smarter models, this will be the ultimate competitive edgeBut there is also a risk of lock-in with platforms that monopolize your contextSimilar to h...
What We're Working On
Just two years ago AI could only retain about 8 pages of informationNow, it can memorize the equivalent of 10 King James BiblesContext window size determines how much models can remember, and it is growing at a staggering rate—1,250x each yearEverything you do, see, or hear now fits as memories in an AI’s context windowCombined with increasingly smarter models, this will be the ultimate competitive edgeBut there is also a risk of lock-in with platforms that monopolize your contextSimilar to h...
