Fatin757/biencoder-v2

modernbert-job-role-matcher

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatin757/biencoder-v2")
# Run inference
sentences = [
    'Accountancy. Senior Internal Audit Assistant Consultant. The Internal Audit Assistant Manager is responsible for developing audit procedures and programmes based on internal audit engagement objectives and scope. He/She is involved in supervising internal audit engagements. The Internal Audit Assistant Manager analyses implications of significant changes and infocomm technology (IT) developments to the organisation, business units and key processes. He also guides the team to uphold professional standards and ensure internal audit engagements are executed in accordance with the International Professional Practices Framework (IPPF). He supports the business by applying data analytics and business intelligence tools to analyse data to interpret findings for business insights. The Internal Audit Assistant Manager is scrupulous and accountable. He exercises due professional care in his work.',
    'Internal Audit Assistant Manager',
    'Sea Transport. Coordinator Sales and Purchase Broker. The Sales and Purchase Broker acts as an intermediary between buyers and sellers of ships and is responsible for overseeing the sale and purchase of ships while ensuring compliance with legal and regulatory requirements. He/She assesses the viability and risks of pursuing new business opportunities and analyses risk management data to highlight potential areas of concern to management. He guides and provides on-the-job coaching to junior colleagues in their daily work.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	1.0

Training Details

Training Dataset

Unnamed Dataset

• Size: 800 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 800 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 96 tokens mean: 181.44 tokens max: 304 tokens	min: 4 tokens mean: 9.15 tokens max: 29 tokens	min: 96 tokens mean: 158.22 tokens max: 279 tokens

• Samples:

  anchor	positive	negative
  Hotel and Accommodation Services. Senior Director of Sales and Marketing. The Director of Sales and Marketing leads the sales and marketing team in optimising revenue from room sales, catering, events and conference services by attaining sales revenue targets. He/She establishes annual business plans, sales and marketing strategies and key performance indicators for the department and drives revenue management. He directs all sales and marketing activities including market research and partnership management. He also directs resource management, sets service guidelines and standards, and establishes ethical and regulatory parameters, procedures and guidelines for sales and marketing activities to ensure the efficient functioning of the department.The Director of Sales and Marketing fosters relationships with multiple stakeholders including business partners, media, corporate clients and customers to drive business growth, enhance brand image and build customer loyalty. As a head of the...	Director of Sales and Marketing	Logistics. Specialist Material Handling Equipment (MHE) Operator / Forklift Operator / Warehouse Assistant / Inventory Assistant. The Material Handling Equipment (MHE) Operator/Forklift Operator/Warehouse Assistant/Inventory Assistant is responsible for sorting, routing and loading cargo to and from various warehousing or storage locations. Systematic and mechanically-inclined, he/she is also responsible for upholding quality standards, ensuring the safe and efficient operation of material-handling equipment and may also be required to support general warehouse operations. He is expected work with internal and external stakeholders to accomplish his work.
  Workplace Safety and Health. Advisor Senior Workplace Safety and Health Officer. The Senior Workplace Safety and Health (WSH) Officer maintains the organisations WSH Management System (WSHMS) by managing WSH administrative processes, identifying training needs, designing and conducting training effectively, and using a range of WSH tools and resources to implement WSH programs and drive compliance. He/She may be expected to supervise a WSH team and work with internal and external stakeholders to accomplish his work.He is analytical and values teamwork and collaboration in order to solve problems.	Senior Workplace Safety and Health Officer	Arts. Associate Associate / Assistant Sound Designer. Associate/Assistant Sound Designers support the planning and development of music, sound effects and soundscapes to be used during a production, to create the desired impact based on the artistic vision. They support Sound Designers in drafting a sound cue list, which describes the changing sounds throughout the entire production, and assist in the creation of the planned sounds and considerations for positioning of speakers. They need to be comfortable with audio and engine tools, as well as sound technologies, and understand various music styles and genres.
  Arts. Associate Associate / Assistant Sound Designer. Associate/Assistant Sound Designers support the planning and development of music, sound effects and soundscapes to be used during a production, to create the desired impact based on the artistic vision. They support Sound Designers in drafting a sound cue list, which describes the changing sounds throughout the entire production, and assist in the creation of the planned sounds and considerations for positioning of speakers. They need to be comfortable with audio and engine tools, as well as sound technologies, and understand various music styles and genres.	Associate / Assistant Sound Designer	Retail. Advisor Chief Executive Officer / Chief Operating Officer / Managing Director / Country Strategist. The Chief Executive Officer/Chief Operating Officer/Managing Director/Country Manager sets overall direction for the organisation, formulates strategic goals and drives organisational growth. He/she identifies new business opportunities, champions the organisation's service excellence aspirations and fosters strategic relationships with stakeholders. He is also responsible for driving the organisations financial, innovation and productivity strategies.He operates in a rapidly transforming business environment where he is accountable for the success of the entire organisation.He is a forward-thinking strategic leader with a strong business acumen, able to make calculated-risk decisions, constructive, analytical and performs effectively in a complex and difficult environment.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 200 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 200 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 96 tokens mean: 187.4 tokens max: 304 tokens	min: 4 tokens mean: 9.09 tokens max: 29 tokens	min: 96 tokens mean: 168.16 tokens max: 258 tokens

• Samples:

  anchor	positive	negative
  Workplace Safety and Health. Officer Senior Workplace Safety and Health Officer. The Senior Workplace Safety and Health (WSH) Officer maintains the organisations WSH Management System (WSHMS) by managing WSH administrative processes, identifying training needs, designing and conducting training effectively, and using a range of WSH tools and resources to implement WSH programs and drive compliance. He/She may be expected to supervise a WSH team and work with internal and external stakeholders to accomplish his work.He is analytical and values teamwork and collaboration in order to solve problems.	Senior Workplace Safety and Health Officer	Arts. Executive Associate / Assistant Sound Designer. Associate/Assistant Sound Designers support the planning and development of music, sound effects and soundscapes to be used during a production, to create the desired impact based on the artistic vision. They support Sound Designers in drafting a sound cue list, which describes the changing sounds throughout the entire production, and assist in the creation of the planned sounds and considerations for positioning of speakers. They need to be comfortable with audio and engine tools, as well as sound technologies, and understand various music styles and genres.
  Accountancy. Senior Internal Audit Assistant Coordinator. The Internal Audit Assistant Manager is responsible for developing audit procedures and programmes based on internal audit engagement objectives and scope. He/She is involved in supervising internal audit engagements. The Internal Audit Assistant Manager analyses implications of significant changes and infocomm technology (IT) developments to the organisation, business units and key processes. He also guides the team to uphold professional standards and ensure internal audit engagements are executed in accordance with the International Professional Practices Framework (IPPF). He supports the business by applying data analytics and business intelligence tools to analyse data to interpret findings for business insights. The Internal Audit Assistant Manager is scrupulous and accountable. He exercises due professional care in his work.	Internal Audit Assistant Manager	Sea Transport. Advisor Sales and Purchase Broker. The Sales and Purchase Broker acts as an intermediary between buyers and sellers of ships and is responsible for overseeing the sale and purchase of ships while ensuring compliance with legal and regulatory requirements. He/She assesses the viability and risks of pursuing new business opportunities and analyses risk management data to highlight potential areas of concern to management. He guides and provides on-the-job coaching to junior colleagues in their daily work.
  Hotel and Accommodation Services. Coordinator Director of Sales and Marketing. The Director of Sales and Marketing leads the sales and marketing team in optimising revenue from room sales, catering, events and conference services by attaining sales revenue targets. He/She establishes annual business plans, sales and marketing strategies and key performance indicators for the department and drives revenue management. He directs all sales and marketing activities including market research and partnership management. He also directs resource management, sets service guidelines and standards, and establishes ethical and regulatory parameters, procedures and guidelines for sales and marketing activities to ensure the efficient functioning of the department.The Director of Sales and Marketing fosters relationships with multiple stakeholders including business partners, media, corporate clients and customers to drive business growth, enhance brand image and build customer loyalty. As a head o...	Director of Sales and Marketing	Logistics. Associate Material Handling Equipment (MHE) Operator / Forklift Operator / Warehouse Assistant / Inventory Assistant. The Material Handling Equipment (MHE) Operator/Forklift Operator/Warehouse Assistant/Inventory Assistant is responsible for sorting, routing and loading cargo to and from various warehousing or storage locations. Systematic and mechanically-inclined, he/she is also responsible for upholding quality standards, ensuring the safe and efficient operation of material-handling equipment and may also be required to support general warehouse operations. He is expected work with internal and external stakeholders to accomplish his work.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• gradient_accumulation_steps: 4
• learning_rate: 2e-05
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	cosine_accuracy
1.0	50	-	3.5529	1.0
2.0	100	14.8274	3.2619	1.0
3.0	150	-	3.2237	1.0

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 4.1.0
• Transformers: 4.52.4
• PyTorch: 2.6.0+cu124
• Accelerate: 1.7.0
• Datasets: 2.14.4
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}