TroubleChute's video: LESS VRAM 8K Tokens HUGE SPEED INCRASE ExLlama for Oobabooga

Oobabooga WebUI had a HUGE update adding ExLlama and ExLlama_HF model loaders that use LESS VRAM and have HUGE speed increases, and even 8K tokens to play around with compared to the previous limit of 2K! This is insanely powerful and will be a huge timesaver for creators, and may even help users with less powerful graphics cards use LLMs! OpenAI Tokenizer: https://platform.openai.com/tokenizer Timestamps: 0:00 - What's new (It's CRAZY!) 0:44 - Open Oobabooga install directory 1:02 - Update Oobabooga WebUI 1:18 - VRAM usage & speed before update (4.3 tokens/s) 1:56 - Fix missing option or update errors 2:33 - Choosing new ExLlama model locader 2:52 - Downloading new model types (8k models) 4:25 - New VRAM & Speed (20 tokens/s! INSANE!) 5:25 - Raise token limit from 2,000 to 8,000+! 7:17 - How many tokens is your text? 7:50 - How long is 8k tokens? 8:45 - EVEN LESS VRAM with ExLlama_HF ----------------------------- 💸 Found this useful? Help me make more! Support me by becoming a member: https://youtube.com/channel/UCkih2oVTbXPEpVwE-U7kmHw/join ----------------------------- 💸 Support me on Patreon: https://patreon.com/TroubleChute 💸 Direct donations via Ko-Fi: https://ko-fi.com/TCNOco 💬 Discuss the video & Suggest (Discord): https://s.tcno.co/Discord 👉 Game guides & Simple tips: https://YouTube.com/TroubleChuteBasics 🌐 Website: https://tcno.co 📧 Need voiceovers done? Business query? Contact my business email: TroubleChute (at) tcno.co ----------------------------- 🎨 My Themes & Windows Skins: https://hub.tcno.co/faq/my-windows/ 👨‍💻 Software I use: https://hub.tcno.co/faq/my-software/ ➡️ My Setup: https://hub.tcno.co/faq/my-hardware/ 🖥️ My Current Hardware: Intel i9-13900k - https://amzn.to/42xQuI1 GIGABYTE Z790 AORUS Master - https://amzn.to/3nHuBHx G.Skill RipJaws 2x(2x32G) [128GB] - https://amzn.to/42cilxN Corsair H150i 360mm AIO - https://amzn.to/42cznvP MSI 3080Ti Gaming X Trio - https://amzn.to/3pdnLdb Corsair 1000W RM1000i - https://amzn.to/42gOTGY Corsair MP600 PRO XT 2TB - https://amzn.to/3NSvwzx 🎙️ My Current Mic/Recording Gear: Shure SM7B - https://amzn.to/3nDGYo1 Audient iD14 - https://amzn.to/3pgf2XK dbx 286s - https://amzn.to/3VNaq7O Triton Audio FetHead - https://amzn.to/3pdjIgZ Everything in this video is my personal opinion and experience and should not be considered professional advice. Always do your own research and ensure what you're doing is safe.