sentdex
sentdex
  • Видео 1 254
  • Просмотров 114 975 905
Building an LLM fine-tuning Dataset
Going through the building of a QLoRA fine-tuning dataset for a language model.
NVIDIA GTC signup: nvda.ws/3XTqlB6
Fine-tuning code: github.com/Sentdex/LLM-Finetuning
5000-step Walls1337bot adapter: huggingface.co/Sentdex/Walls1337bot-Llama2-7B-003.005.5000
WSB Dataset: huggingface.co/datasets/Sentdex/WSB-003.005
"I have every reddit comment" original reddit post and torrent info: www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/
2007-2015 Reddit Archive.org: archive.org/download/2015_reddit_comments_corpus/reddit_data/
Reddit BigQuery 2007-2019 (this has other data besides reddit comments too!): reddit.com/r/bigquery/comments/3cej2b/17_billion_reddit_com...
Просмотров: 39 524

Видео

Visualizing Neural Network Internals
Просмотров 41 тыс.4 месяца назад
Visualizing some of the internals of a neural network during training and inference. Starting and full code: github.com/Sentdex/neural-net-internals-visualized Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/sentdex/ Support the content: pythonprogramming.net/support-donate/ Twit...
Getting Back on Grid
Просмотров 18 тыс.4 месяца назад
Establishing an internet connection in an internet desert, then figuring out (well, starting to) networking. Combined with Starlink as my internet provider, I ended up going with a wifi bridge implementation with a couple of Ubiquiti nanostation AC locos to network between buildings at 100 meters of distance. The Ubiquiti units can also do point to point (ptp), but so far the wifi bridge setup ...
Open Source AI Inference API w/ Together
Просмотров 31 тыс.6 месяцев назад
Exploring the Together Inference API (www.together.ai/) Together API basics jupyter notebook examples: github.com/Sentdex/Together-API-Basics Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/sentdex/ Support the content: pythonprogramming.net/support-donate/ Twitter: s...
INFINITE Inference Power for AI
Просмотров 25 тыс.6 месяцев назад
Testing and enjoying the Comino Grando Server machine with 6x RTX 4090s from Comino (www.comino.com/) Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/sentdex/ Support the content: pythonprogramming.net/support-donate/ Twitter: sentdex Instagram: sentdex ...
Pandas Dataframes on your GPU w/ CuDF
Просмотров 41 тыс.7 месяцев назад
An overview and some quick examples of using CuDF's Pandas accelerator and how much faster it can be than vanilla Pandas for data analysis. Colab demo of Rapids: nvda.ws/3LWggQj AI and Data Science Virtual Summit: nvda.ws/3ZR3wjL Notebook in this video: gist.github.com/Sentdex/469c30385d06719519af13125db85edc Install CuDF: pip install cudf-cu11 extra-index-url=pypi.nvidia.com (or cu12) Neural N...
QLoRA is all you need (Fast and lightweight model fine-tuning)
Просмотров 65 тыс.9 месяцев назад
Learning and sharing my process with QLoRA (quantized low rank adapters) fine-tuning. In this case, I use a custom-made reddit dataset, but you can use anything you want. I referenced a LOT of stuff in this video, I will do my best to link everything, but let me know if I forget anything. Resources: WSB-GPT-7B Model: huggingface.co/Sentdex/WSB-GPT-7B WSB-GPT-13B Model: huggingface.co/Sentdex/WS...
Chat Interface for your Local Llama LLMs
Просмотров 20 тыс.10 месяцев назад
A tutorial of sorts covering how to create streaming chat interfaces using Gradio for the various chat/instruct large language models from HuggingFace. Sample code: huggingface.co/spaces/Sentdex/StableBeluga-7B-Chat/blob/main/app.py Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r...
Gzip is all You Need! (This SHOULD NOT work)
Просмотров 150 тыс.10 месяцев назад
Github code: github.com/Sentdex/Simple-kNN-Gzip Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/sentdex/ Support the content: pythonprogramming.net/support-donate/ Twitter: sentdex Instagram: sentdex Facebook: pythonprogramming.net/ Twitch: ...
Better Attention is All You Need
Просмотров 62 тыс.11 месяцев назад
Addressing the current state of attention for artificial intelligence and why it's currently holding back maximum context lengths. Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/sentdex/ Support the content: pythonprogramming.net/support-donate/ Twitter: sentdex Inst...
The BEST Open Source LLM? (Falcon 40B)
Просмотров 98 тыс.11 месяцев назад
TII Call for Proposals with Falcon 40B: falconllm.tii.ae/proposal.php Falcon Github samples: github.com/Sentdex/Falcon-LLM TermGPT: ruclips.net/video/O4EmRi0_CI4/видео.html GPT-4 Overview: ruclips.net/video/lJNblY3Madg/видео.html Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/se...
OpenAI GPT-4 Function Calling: Unlimited Potential
Просмотров 227 тыс.Год назад
Function calling is a new capability for OpenAI's GPT-4 and GPT-3.5 via the API. Function-calling allows you to extract structured outputs from the GPT model. Github notebook: github.com/Sentdex/ChatGPT-API-Basics/blob/main/function_calling.ipynb Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: ww...
Letting GPT-4 Control My Terminal (TermGPT)
Просмотров 74 тыс.Год назад
Giving LLMs like GPT-4 the ability to plan and execute terminal commands. TermGPT github: github.com/Sentdex/TermGPT/ OpenAI Chat API tutorial: github.com/Sentdex/ChatGPT-API-Basics Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/sentdex/ Support the content: pythonprogramming.ne...
Building an Open Assistant API
Просмотров 30 тыс.Год назад
Working with one of the Open Assistant models, a 12B parameter Pythia model (huggingface.co/OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5) Github: github.com/Sentdex/OpenAssistant_API_Pythia_12B Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/channel/UCfzlCWGWYyIQ0aLC5w48gBQjoin Discord: discord.gg/sentdex Reddit: www.reddit.com/r/sentdex/ Support the content: python...
Sparks of AGI? - Analyzing GPT-4 and the latest GPT/LLM Models
Просмотров 58 тыс.Год назад
An in-depth look into the current state of the art of Generative Pre-trained Transformer (GPT) language models, with a specific focus on the advancements and examples provided by OpenAI in their GPT4 Technical Report (arxiv.org/abs/2303.08774) as well as the Microsoft "Sparks of AGI" Paper (arxiv.org/abs/2303.12712). Neural Networks from Scratch book: nnfs.io Channel membership: ruclips.net/cha...
ChatGLM: The ChatGPT killer? Checking out ChatGLM6B
Просмотров 102 тыс.Год назад
ChatGLM: The ChatGPT killer? Checking out ChatGLM6B
GPT Journey - A text and image game with ChatGPT
Просмотров 35 тыс.Год назад
GPT Journey - A text and image game with ChatGPT
ChatGPT API in Python
Просмотров 178 тыс.Год назад
ChatGPT API in Python
Image Editing A.I.
Просмотров 21 тыс.Год назад
Image Editing A.I.
The AI wars: Google vs Bing (ChatGPT)
Просмотров 109 тыс.Год назад
The AI wars: Google vs Bing (ChatGPT)
ChatGPT Writes a Chatbot AI
Просмотров 188 тыс.Год назад
ChatGPT Writes a Chatbot AI
OpenAI's ChatGPT is a MASSIVE step forward in Generative AI
Просмотров 470 тыс.Год назад
OpenAI's ChatGPT is a MASSIVE step forward in Generative AI
Google A.I. Diffusion Image Editing w/ Prompt to Prompt
Просмотров 42 тыс.Год назад
Google A.I. Diffusion Image Editing w/ Prompt to Prompt
Google's DreamFusion AI: Text to 3D
Просмотров 200 тыс.Год назад
Google's DreamFusion AI: Text to 3D
Open AI’s Whisper is Amazing!
Просмотров 479 тыс.Год назад
Open AI’s Whisper is Amazing!
The Future of User Interfaces with A.I.
Просмотров 70 тыс.Год назад
The Future of User Interfaces with A.I.
Creating Stable Diffusion Interpolation Videos
Просмотров 54 тыс.Год назад
Creating Stable Diffusion Interpolation Videos
Exploring an AI’s Imagination (Stable Diffusion and MidJourney)
Просмотров 146 тыс.Год назад
Exploring an AI’s Imagination (Stable Diffusion and MidJourney)
$5 MILLION AI for FREE
Просмотров 426 тыс.Год назад
$5 MILLION AI for FREE
Does a Deep Learning Laptop Exist? - Tensorbook Review
Просмотров 50 тыс.Год назад
Does a Deep Learning Laptop Exist? - Tensorbook Review

Комментарии

  • @Veptis
    @Veptis День назад

    What happened to the diffusion based remake?

  • @davidpolycarp8803
    @davidpolycarp8803 День назад

    Where can I learn tensorflow if you have a site or somewhere you recommend

  • @Harshit-qc5ck
    @Harshit-qc5ck День назад

  • @Harshit-qc5ck
    @Harshit-qc5ck 2 дня назад

  • @ARK-ct4ww
    @ARK-ct4ww 2 дня назад

    me watching the tutorial in 2024 the best ever tutorial

  • @Dragonfly-jk7wo
    @Dragonfly-jk7wo 2 дня назад

    Thanks for your tutoria, but the data transfer should have be done quicker if you umount this storage from the first CPU server and then mount it on your GPU server? (Update: Found this solution at the end of your video.)

  • @mikecripps2011
    @mikecripps2011 2 дня назад

    Thanks!

  • @spysec6525
    @spysec6525 3 дня назад

    After years 🤣

  • @tkumardalai9995
    @tkumardalai9995 4 дня назад

    its a nice explanation and a good hack

  • @gabrielcampos295
    @gabrielcampos295 4 дня назад

    i could not find a video that explained as well as this for the past 2 months, this is gold, congrats, you're amazing!

  • @prateekyadav9811
    @prateekyadav9811 4 дня назад

    Harrison, please inform us if you plan to continue the tutorial series for NNFS. It's a brilliant piece of work. Please do tell us either way. You been super quiet on this for long :P Mighty thanks for your content!

  • @prateekyadav9811
    @prateekyadav9811 5 дней назад

    Bhai please continue this series!! It's so bloody awesome.

  • @prateekyadav9811
    @prateekyadav9811 5 дней назад

    Why don't we clip the predicted values as 1e-7 to 1. I get why we don't add 1e-7 to 1 to give the max possible value cause then the negative log will be negative and negative loss doesn't mean anything. But why subtract 1e-7 from 1 instead? Why not we keep it as 1?

  • @prateekyadav9811
    @prateekyadav9811 5 дней назад

    So few comments compared to the views! Suggests that people aren't really following through. Sad. This is gold!

  • @sevdattufanogullari6581
    @sevdattufanogullari6581 5 дней назад

    Why is the bias added to the end of the formula?

  • @igorson7929
    @igorson7929 5 дней назад

    4:54, actually no output after Softmax activation function can be 0, because when you softmax you take euler number to power of your input, and you can never get 0 out of this unless you take e to power of negative infinity, but I assume that your input is never negative infinity. Then when you normalize the final output you also cannot get 0 out of this.

    • @prateekyadav9811
      @prateekyadav9811 5 дней назад

      I am guessing its possible due to lack of computational power for the lack of a better term. We might obtain a number to the order of negative 40 that might be rounded off to 0 after normalization. Just a guess. Your comment made me wonder.

  • @Moai_rocc
    @Moai_rocc 6 дней назад

    9th grade me trying to make a bot because I couldn't make a good economic simulation when I dictated when someone did something edit: I may have no idea what the weird E / W rotated 90 degrees is

  • @SIDAMKRISHNA-sf7jk
    @SIDAMKRISHNA-sf7jk 6 дней назад

    you got a new cup for every video and what do you drink

  • @yugandharsurya
    @yugandharsurya 6 дней назад

    Please upload other videos 🥹🥹

  • @Ed-ix2vk
    @Ed-ix2vk 6 дней назад

    is numpy 2 ok?

  • @prateekyadav9811
    @prateekyadav9811 7 дней назад

    Absolutely love this series! I have a question: how did we arrive at fitting functions using NN? I thought we use NN for classification. And when fitting to the sine function, what would input data be like? What would be its features? Just one value i.e. x?

  • @SIDAMKRISHNA-sf7jk
    @SIDAMKRISHNA-sf7jk 7 дней назад

    i never expect you to be a car enthusiastic POV : man with actual skills

  • @woojay
    @woojay 8 дней назад

    Thank you so much.

  • @mohammedsufyanrizvi2595
    @mohammedsufyanrizvi2595 8 дней назад

    Why is this not complete????????????

  • @laithovkh8303
    @laithovkh8303 8 дней назад

    if you are here to learn the coding for Neural network, I am advising you to leave now and dont waste your time like we did he just selling his stupid book this series are unfinished for years and there are now launching nor backpropagation coding it is all a scam, try others like 3blue one brown and good luck.

  • @laithovkh8303
    @laithovkh8303 8 дней назад

    i spent 3 hours watching this just to realize this dude does not know also how neural network works, just changed the subject and made different types of random videos.

  • @The_Quaalude
    @The_Quaalude 8 дней назад

    Thank you for explaining this concept with actual python code ‼️🔥🔥🔥

  • @BatteryProductions
    @BatteryProductions 8 дней назад

    you need client socket instead of just s.send so that s socket can keep listening for more connections.. remember the 5 queue.. well if 5 connections come, those connections get passed to 5 client socket objects so s can then go get other 5 connections...

  • @sagarpanwar546
    @sagarpanwar546 8 дней назад

    How can I order the Hard copy of the Book ?

  • @JusticeNDOU
    @JusticeNDOU 9 дней назад

    you cannot DOS someone if you do not await, because once you do not await you probably never even had a coroutine or you have a coroutine but its not sent to the event loop, the effect in eiher way is that you end up with an awaitable which is never executed

  • @alessandrocoppelli3056
    @alessandrocoppelli3056 9 дней назад

    hello,i'm trying to use PPO and A2C for my discrete-box environment. i have set negative rewards in order to teach the agent to avoid impossible operation in my environment. most of the training time is spent to learn to avoid those operations with negative rewards. Is there a method to directly "tell" the agent (inside the agent itself) to avoid those operations, instead of spend training time? thanks in advance

  • @Harshit-qc5ck
    @Harshit-qc5ck 10 дней назад

  • @Harshit-qc5ck
    @Harshit-qc5ck 10 дней назад

  • @Ukraine-0
    @Ukraine-0 10 дней назад

    WTF MAN WHY DID I STUDY VERTEXES AND JOINTS AND ALL THAT STUFF?!

  • @Codingskills-ne2lh
    @Codingskills-ne2lh 11 дней назад

    Hey man looks like the command "py -3.7" is not working for me. I doo have python 3.12 installed and when I check the list of environments it shows 3.12 and 3.8. How can I add 3.7 here. Should I go and add it in the env path and downloading python 3.7?

  • @michaelmoore7568
    @michaelmoore7568 11 дней назад

    Is Sentdex the best Python coder in the world?

  • @snipplerifsky
    @snipplerifsky 13 дней назад

    4 years later and still one of the best series on the subject

  • @helloeveryone6614
    @helloeveryone6614 13 дней назад

    Thankyou bro❤

  • @Anandgamerzzz-ru8xt
    @Anandgamerzzz-ru8xt 14 дней назад

    Get an error string indices must be integers, not 'str'

  • @rachaelkibicho7461
    @rachaelkibicho7461 14 дней назад

    Can anyone access his python shell scripts? I'm having trouble with the input at the attribute category_slug. Were we not supposed to fill that attribute at the python shell. I feel lost

  • @Harshit-qc5ck
    @Harshit-qc5ck 14 дней назад

  • @imad-ice
    @imad-ice 14 дней назад

    why is this resource being underrated, people nowadays import tensorflow, and PyTorch, -> say u build a neural network, but this man combined theory and lab in such a way that it is easy to understand

  • @Uncreeperble
    @Uncreeperble 14 дней назад

    "You won't get through this in a weekend" Challenge accepted.

  • @No7es-Txken
    @No7es-Txken 14 дней назад

    Loved the videos, really helped me understand Neural Networks deeper. I’m looking forward to buying the book but I also enjoy the videos that you do. Are you going to continue video support for this series or is the book my only way forward from here?

  • @norimashiron3635
    @norimashiron3635 14 дней назад

    for row in range(len(game)): diag1.append(game[row][row]) diag2.append(game[row][len(game)-1-row])

  • @souravdey1227
    @souravdey1227 15 дней назад

    Your videos always feel so grounded. Most other videos on such topics make me feel so insignificant. Also, I have a friend, Zeke, from Florida, and you two are soo uncannily similar!!!!

  • @namitshah9535
    @namitshah9535 16 дней назад

    That ain't a home lab it feels like a tiny datacenter

  • @proflead
    @proflead 16 дней назад

    Epic video! :)

  • @Eisenwolf.Ironwulf
    @Eisenwolf.Ironwulf 16 дней назад

    Still here ... still waiting ... with my book and my unwavering Will to wait

    • @DawitMengistuAbajifar
      @DawitMengistuAbajifar 10 дней назад

      I can't afford to buy the book? Any way you can sent it to me?

    • @Eisenwolf.Ironwulf
      @Eisenwolf.Ironwulf 10 дней назад

      @@DawitMengistuAbajifar Sorry, but i cant give my copie away, I still need it. But if you cant afford it, ask sentdex, mby he can give you a code or something. Thats far better than pirating it.

    • @DawitMengistuAbajifar
      @DawitMengistuAbajifar 10 дней назад

      @@Eisenwolf.Ironwulf alright, found it anyway.

  • @rajvaidya1691
    @rajvaidya1691 17 дней назад

    this is going over my head now