People is taking it lightly what the world is observing right now. In all these fuss of ๐ก๐๐ถ๐ฑ๐ถ๐ฎ, ๐ ๐ฒ๐๐ฎ and ๐ข๐ฝ๐ฒ๐ป๐๐, the ๐ฎ๐ฟ๐๐ถ๐ณ๐ถ๐ฐ๐ถ๐ฎ๐น ๐ถ๐ป๐๐ฒ๐น๐น๐ถ๐ด๐ฒ๐ป๐ฐ๐ฒ universe is taking a new direction, what I have been ๐ต๐ฆ๐ญ๐ญ๐ช๐ฏ๐จ ๐ง๐ฐ๐ณ ๐ข ๐บ๐ฆ๐ข๐ณ ๐ฏ๐ฐ๐ธ.
๐ก๐๐ถ๐ฑ๐ถ๐ฎ just released RTX 5090 which is a giant beast. However this yearโs least performance chip (RTX5070) was as powerful as last yearโs best chip (RTX4090) and at ๐ญ/๐ฏ๐ฟ๐ฑ ๐ผ๐ณ ๐ถ๐๐ ๐ฝ๐ฟ๐ถ๐ฐ๐ฒ!! ๐ง๐ต๐ถ๐ ๐ถ๐ ๐ฐ๐ฟ๐ฎ๐๐! Raw computing power and resources are getting easily available now compared to what it was 3 years back and itโs not gonna stop anytime soon.
So, using better computing power to get ๐ต๐ถ๐ด๐ต๐ฒ๐ฟ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ถ๐ป๐ด ๐บ๐ผ๐ฑ๐ฒ๐น is not a novelty now. Anyone can add up a more complex model, get more parameters and train harder to build a greater model! But it also increases the cost. This is the reason why we have to shift our focus from general models to ๐๐ฎ๐๐ธ ๐๐ฝ๐ฒ๐ฐ๐ถ๐ณ๐ถ๐ฐ ๐น๐ผ๐-๐ฐ๐ผ๐๐ ๐๐ ๐. Except for conversational AI we donโt need that much flexibility. Itโs time to distribute work among SLMs. Instead of ๐ญ๐ฌ๐ฌ๐ ๐ผ๐ณ ๐ฏ๐ถ๐น๐น๐ถ๐ผ๐ป๐ letโs get back to ๐ณ-๐ด๐ฌ๐ฌ๐ ๐ฝ๐ฎ๐ฟ๐ฎ๐บ๐ฒ๐๐ฒ๐ฟ๐ or maybe ๐ฎ ๐ณ๐ฒ๐ ๐ฏ๐ถ๐น๐น๐ถ๐ผ๐ป๐. The way itโs moving, expensive and powerful GPUs will keep coming but that doesnโt count in AI boom. True AI needs to come back with feature engineering, correlation and so on. A lot of recent papers show achieving near ๐๐๐ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ฎ๐ป๐ฐ๐ฒ ๐๐ถ๐๐ต ๐ฆ๐๐ ๐.
๐ ๐ฒ๐๐ฎ just launched its Llama 3.3 model of 70B parameters which achieves near-equal results to Llama 3.1 which is of 405B parameters. This is an almost ๐ด๐ฏ% ๐ฝ๐ฎ๐ฟ๐ฎ๐บ๐ฒ๐๐ฒ๐ฟ ๐ฟ๐ฒ๐ฑ๐๐ฐ๐๐ถ๐ผ๐ป to achieve the similar scale result! Just a building block towards what the future holds. The world is not anymore at the spot where compute used to be the limiting factor. Hence, focusing on ๐ณ๐๐ป๐ฑ๐ฎ๐บ๐ฒ๐ป๐๐ฎ๐น๐ ๐ผ๐ณ ๐๐, solving problems with research and engineering skills is much more important again.
Sources:
