SOLAR 10.7B Uncensored

Community (based on Upstage SOLAR) · February 2024

activeOpen Weightdecoder onlytext
Parameters10.7B
Context Window4K tokens

Description

Uncensored variant of Upstage's SOLAR 10.7B model, originally developed by a Korean AI company. The base model was built using a novel technique called Depth Up-Scaling (DUS), which takes a smaller pre-trained model and expands it by duplicating and merging its internal layers to create a larger, more capable model without training from scratch.

Key Innovations

Abliteration
AbliterationRemoving safety guardrails from a model through targeted fine-tuning or weight manipulation. Controversial but popular in open-source community.
Open Weight
Open WeightModel weights are publicly released but training data/code may not be. Enables fine-tuning but not full reproduction.