Sebastian Gabarain's picture

Sebastian Gabarain

Locutusque

·

SebastianG74019

AI & ML interests

Pushing performance in small language models

Organizations

Locutusque's activity

upvoted a paper 1 day ago

Chameleon: Mixed-Modal Early-Fusion Foundation Models

Paper • 2405.09818 • Published 12 days ago • 92

upvoted a paper 6 days ago

LoRA Learns Less and Forgets Less

Paper • 2405.09673 • Published 13 days ago • 71

upvoted a collection 16 days ago

Yi-1.5 (2024/05)

10 items • Updated 9 days ago • 75

upvoted an article 20 days ago

Article

Can we create pedagogically valuable multi-turn synthetic datasets from Cosmopedia?

By

•

21 days ago

• 6

upvoted 2 articles about 1 month ago

Article

Introducing the Open Chain of Thought Leaderboard

Apr 23

• 20

Article

Fine-tune Llama 3 with ORPO

By

•

Apr 22

• 191

upvoted a collection about 1 month ago

Meta Llama 3

This collection hosts the transformers and original repos of the Meta Llama 3 and Llama Guard 2 releases • 5 items • Updated Apr 18 • 543

upvoted a paper about 1 month ago

Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck

Paper • 2404.07647 • Published Apr 11 • 4

upvoted a collection about 1 month ago

OpenCerebrum-2.0

My open source take on Aether Research's proprietary Cerebrum dataset. • 3 items • Updated Apr 13 • 1

upvoted 2 papers about 2 months ago

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

Paper • 2404.05961 • Published Apr 9 • 62

Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

Paper • 2404.02258 • Published Apr 2 • 101

upvoted a paper 2 months ago

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Paper • 2403.09629 • Published Mar 14 • 54

upvoted 2 collections 3 months ago

Augmentable

A collection of datasets that should be augmented further with gpt-4 • 13 items • Updated Jan 2 • 4

Hub Models

273 items • Updated 3 days ago • 4

upvoted 5 collections 4 months ago

Merges

Experimental LLM merging • 1292 items • Updated 3 days ago • 7

Qwen1.5

Qwen1.5 is the improved version of Qwen, the large language model series developed by Alibaba Cloud. • 55 items • Updated 16 days ago • 177

ZeroGPU Spaces

ZeroGPU Spaces made by the community • 16 items • Updated 11 days ago • 177

Medical Evaluation Datasets

39 items • Updated Apr 24 • 4

Tiny Series

Tiny datasets that empower the foundation of Small Language Model! • 11 items • Updated Jan 26 • 31

upvoted a collection 5 months ago

Pretrained Text-Generation Models Below 250M Parameters

Great candidates for fine-tuning targeting Transformers.js, ordered by number of parameters. • 7 items • Updated 15 days ago • 6

upvoted a paper 5 months ago

WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation

Paper • 2312.14187 • Published Dec 20, 2023 • 49

upvoted 2 collections 5 months ago

smol llama

🚧"raw" pretrained smol_llama checkpoints - WIP 🚧 • 4 items • Updated 30 days ago • 5

Trained Models 🏋️

They may be small, but they're training like giants! • 8 items • Updated 15 days ago • 14

upvoted a paper 5 months ago

LLM in a flash: Efficient Large Language Model Inference with Limited Memory

Paper • 2312.11514 • Published Dec 12, 2023 • 253

upvoted a collection 5 months ago

InstructWise

InstructWise is a series of model created to act as helpful virtual assistant while maintaing the memory efficiency. • 2 items • Updated Dec 3, 2023 • 2