Project AURIONAI Use-Cases Research & Innovation
language encoder

DistilRoBERTa base

DistilRoBERTa base, a checkpoint of the DistilBERT family. Smaller, faster BERT made by knowledge distillation during pre-training, keeping most of BERT's accuracy for on-device and low-cost text tasks.

Hugging FaceHugging FaceUSA / FranceDistilBERTFamily
Hugging Face
Levelvariant
Classlanguage encoder
Parameters82.76M
Contextnot in the notes
Releasednot in the notes
Opennessopen weights permissive
Runs onnot in the notes
Statusactive

R&D Topics

Compact text encoders through distillationproduced by

Identity

Developer
Hugging Face
Family
DistilBERT
Released
– sources
Weights published
–
Record checked
12 Oct 2026
Where this comes from
Developer and dates from the research notes. Release date is the vendor's public announcement; weights date is when open weights were published.

Architecture and Size

Architecture
dense transformer
Total parameters
82.76M sources
Active parameters
–
Checkpoint parameters
–
Note
Parameter count from the safetensors index on Hugging Face.
Context length
not in the notes
Input
text
Output
text
Where this comes from
Parameter counts as the developer states them. Where the checkpoint holds a different total, both are shown (register A16).

Openness and Access

Openness
open weights permissive
Licence
Apache License 2.0 sources
Access routes
open weights download
Where this comes from
The licence file governs over registry tags and aggregator labels.

Deployment

Runs on
not in the notes
Footprint
not in the notes
Runtimes
not in the notes
Where this comes from
Deployment tiers and runtimes as named in the notes. Footprint is the notes' figure, not a measurement by AURION.

Indonesia

Indonesian support
not stated
Regional languages
not in the notes
Jakarta hosting
unknown
Languages
Model card languages: en
Where this comes from
Indonesia fields are carried on every model; unknown is an allowed value.

Benchmark Results

The notes record no benchmark result for this model.

Lineage

The notes record no lineage for this model.

Industry Use Cases

No industry use case links this model.

Sources

  1. arxiv.org/abs/1910.01108 https://arxiv.org/abs/1910.01108
  2. huggingface.co/distilbert/distilroberta-base https://huggingface.co/distilbert/distilroberta-base
  3. jumpstart-cache-prod-us-west-2.s3.us-west-2.amazonaws.com/models_manifest.json https://jumpstart-cache-prod-us-west-2.s3.us-west-2.amazonaws.com/models_manifest.json
  4. aihub.qualcomm.com/models/distil_bert_base_uncased_hf https://aihub.qualcomm.com/models/distil_bert_base_uncased_hf

To cite this page: Project AURION, AI Use-Cases Research & Innovation, "DistilRoBERTa Base", https://aurion.semai.id/models/hugging-face-distilroberta-base/, records of October 2026.