Meetups/Infra/2025-01-13
Jump to navigation
Jump to search
Noisebridge | About | Visit | 272 | Manual | Contact | Guilds | Resources | Events | Projects | WGs | 5MoF | Meetings | Donate | V · T · E |
Events | Hosting | Streaming | Meetup | Upcoming Events | Anniversaries | Hackathons | External Events | V · T · E |
Meetups / Infra: 2025 | Template | Pad | Jitsi | V · T · E |
Big meeting, trying out LLM/ML themed meeting. Discussions of LLM & CLI tools. Discussion of topics for future meetings.
Introductions[edit | edit source]
- [name] - [background]. [goals for meetup, or interests to explore]
- Loren - host meetups to talk about infra -- to keep sharp, teach, learn -- topics for future meetings & some cli llm tools
- Doug - - learn from people who know LOTS about computer. Happy to talk about absolutely anything
- Anup - software engineer - now not at a company, learning about tools and techniques to spin out a company
- Wolf - he/him - run my own infra for a long item. Focus on systems that are simple enough to be [forgettably] understandable. Playing with duckdb system today, re: parquet
- Erik - working today to get system up, bare metal cloud -- interested in getting the desktop set up
- Devinder -- tinker with this stuff, swe/ consultant
- Yuri - work on search engine, integrated with infrastructure.
- Izin (sp) - recent econ/cs grad, want to learn more about software
- David - data engineer, here for python class - check this out first
- Greg - currently unemployed programmer. Run lots of self-hosted computing infrastructure.
- Jake - do [] at work, video games on the side. Not much experience in LLM stuff, more in game tree stuff.
- Sameer -- generalist full-stack engineer, worked in front-/back-end and SRE
- Michael - work on gpu software at job -- interested in: this, using llm in terminal
- Kevin - like writing code, dabble in llm related stuff a lot lately, like many
Lesson or Demo[edit | edit source]
- LLM - command line tool for interacting with large language models
- infra meetup tool -- record, transcribe, pastebin, feed of posts, chat (multi-threaded)
- echoeval
- ddgr -- duckduckgo search
- llamafile -- libcosmo / jart/justine-tunney -
- open language models & closed, foundational language models
- Kevin - mcp -- model-context-protocol
- anthropic mcp Nov 2024.
- location data
- Topics for ML/LLM -
- Papers (8 ...) --
- deekseek
- how to to be fooled by papers. gwern -- (8 + 4 /2)
- RAG development: -- MCP -- (8)
- Use cases: what do people want to do? (6)
- RAG versus / with fine-tuning.
- MCP --
- intro to MCP ++ 10
- llms.txt -- https://llmstxt.site/ -- 8+ (moderately strongly against it)
- MCP --
- much more high-performing speech-to-text. cumbersome workflows -- 4+ (demo promised)
- llm + home-automation (6+)
- Sam S / wandb -- calls him to talk about productivity & blockers (5)
- V--ki? -- rewind (?)
- Divinder -- cursor.sh & Windsurf talk -- IDE 6+
- Sameer -- uithub.com (s/g/u/) -- llms.txt dump of repo --
- way that we're doing software -- with llm tooling --
- Papers (8 ...) --
- AlphaFold -- Mike / computational chemistry
(funny story: uk friends, asking alexa to add to shopping list at friends houses)
Wolf, book, related: How to [wrech] [a] [nice] [beach] -- Recognize speech. https://www.amazon.com/How-Wreck-Nice-Beach-Vocoder/dp/1612190928
- arxiv?
- https://arxiv.org/abs/2412.19437 -- DeepSeek paper
- https://arxiv-sanity-lite.com/ -- Karpathy's search/watching tool
- DataColada, Richard McAlrath-- script kiddies, not reproducible
- Youtube series: https://www.youtube.com/@rmcelreath
Causal Inference Statistical Rethinking textbook. R lang course. David MacKay inference.co.uk
- Rest in peace: David MacKay, Ross Anderson, Max Chiswick,
Takeway requests[edit | edit source]
- throw a few bucks noisebridge way Donate
- RSVP to luma to get future invites, feedback there. https://lu.ma/noisebridge
- post in discord/dm me/email (nb at mcint?) -- about topics you want to see Discord #meetup-infra
- help build tools for meetup hosting: transcripts, pastebin, chat (multi-thread)
- Comments & voting -- https://pol.is/3dmuc7ndsn
- Read aloud: clarify for meetup. We are taking notes in a riseup pad (or I am--help appreciated, and links). We have meeting notes posted to the wiki. noisebridge.net, search Infra, or Meetups/Infra. (the Infrastructure page has a disambiguation link.)
- Shell, web services, self-hosting, networking!
Questions, Discussion, or Coworking[edit | edit source]
- [Issue]
For next time[edit | edit source]
- Loren's tasks:
- make a laser-cut, etched acrylic sign with event links & QR code
- links in lu.ma. + nb.wtf notes link
Questions[edit | edit source]
Readings & Exercises[edit | edit source]
- Readings
- Exercises
Join online[edit | edit source]
- Try it yourself!
- Join libera.chat #nb-meetup-infra
https://www.noisebridge.net/wiki/Meetups/Infra
NOTES on RAG feel free to delete this, but i made a query on RAG and how it works. The big diff from tuning being you're not affecting any weights of the model and since you're doing RAG at inference time, it ought to be much cheaper. * Prepare the Knowledge Base (System Setup): What happens: External data (e.g., documents, articles, or FAQs) is collected, preprocessed, and converted into vector embeddings using an embedding model (e.g., OpenAI's embeddings or Sentence Transformers). Who does this: Engineers or automated pipelines. When: During the initial setup or whenever new data is added to the knowledge base. How: The embeddings are stored in a vector database (e.g., FAISS, Pinecone, or ElasticSearch) for efficient similarity search. * Process the User Query (Input Handling): What happens: The user submits a query. The system transforms this query into a vector representation using the same embedding model used during data preparation. Who does this: The retrieval system (middleware connecting user input to the knowledge base). When: Immediately after the user submits a query. How: The system generates a query embedding that captures its semantic meaning. * Retrieve Relevant Data (Search Step): What happens: The system sends the query embedding to the vector database, which finds the most similar stored embeddings and retrieves corresponding documents. Who does this: The vector database performs the search, returning ranked results (documents or snippets). When: Right after the query embedding is generated. How: Similarity metrics like cosine similarity or dot product measure the relevance of documents to the query. * Augment the Query (Input Fusion): What happens: The retrieved documents are combined with the original user query to provide the LLM with more context. This step enhances the input for response generation. Who does this: The system middleware or pre-processing layer prepares the augmented input (e.g., concatenates the documents with the query in a predefined format). When: Before sending the input to the LLM for processing. How: By appending or embedding the retrieved content into the LLM's input prompt. * Generate the Response (LLM Processing): What happens: The LLM uses the augmented input (query + retrieved data) to generate a response that is both accurate and contextually relevant. Who does this: The LLM processes the enhanced input and produces the final output for the user. When: After receiving the augmented query. How: The LLM treats the retrieved information as additional context, seamlessly integrating it into its natural language generation process.