- Is quantization allowed? Yes!
- Is distillation allowed? The base model shall be the primary model in the inference of the submitted model. As such, participants could use distillation only if the base model is the student model.
- Is finetuning on the evaluation allowed? Yes finetuning is allowed. The participants will not have access to the database that will be used for the evaluation.
- Can we deliver our solutions in container? For evaluation, the organizing team will use the commands ‘vllm serve –config vllm_config.yaml’ and ‘llama-server -hf model_hf’ to perform the evaluation. As long as the submission work with these commands and the base model is the primary model in the inference of the submitted model, the submission will be allowed.
- What is the best solution to deliver my solution on Hugging Face? You will need to upload (i) model weights (ii) README with information on the process you have applied to the model (iii) a config file for vllm or llama.
The organisation HuggingFace account to be shared on the submission day
Should you have any further technical questions, please contact the evaluation team: resilientchallenge2026@peren.gouv.fr
