Model reference · open weights
CoCoSoDa is an open-weight embedding model from DeepSoftwareAnalytics. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | DeepSoftwareAnalytics |
|---|---|
| Type | Embedding models |
| Task | Embeddings |
| Context | 1026 tokens |
| Runs with | transformers |
| Released | 2023-02-07 |
| Popularity | 524 downloads / month |
| Licence | Unknown |
About
Our approach adopts the pre-trained model as the base code/query encoder and optimizes it using multimodal contrastive learning and soft data augmentation.
CoCoSoDa is comprised of the following four components:
Pre-trained code/query encoder captures the semantic information of a code snippet or a natural language query and maps it into a high-dimensional embedding space. as the code/query encoder.
Momentum code/query encoder encodes the samples (code snippets or queries) of current and previous mini-batches to enrich the negative samples.
Soft data augmentation is to dynamically mask or replace some tokens in a sample (code/query) to generate a similar sample as a form of data augmentation.
Multimodal contrastive learning loss function is used as the optimization objective and consists of inter-modal and intra-modal contrastive learning loss. They are used to minimize the distance of the representations of similar samples and maximize the distance of different samples in the embedding space.
import torch
from transformers import RobertaTokenizer, RobertaConfig, RobertaModel
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = RobertaTokenizer.from_pretrained("DeepSoftwareAnalytics/CoCoSoDa")
model = RobertaModel.from_pretrained("DeepSoftwareAnalytics/CoCoSoDa")
Shi, E., Wang, Y., Gu, W., Du, L., Zhang, H., Han, S., ... & Sun, H. (2022). CoCoSoDa: Effective Contrastive Learning for Code Search. ICSE2023.
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys cocosoda for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (cocosoda below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"cocosoda","input":"text to embed"}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.