Core Technologies

• AVX-512 Optimization

AVX-512 (Advanced Vector Extensions) is a set of CPU instructions that accelerate vector processing, boosting computational throughput. Marsha utilizes AVX-512 to enhance performance, enabling CPU-based inference tasks to achieve speeds approaching those of mid-range GPUs, significantly improving efficiency for AI workloads in low-power systems.

• AMX-TILE

The AMX-TILE (Advanced Matrix Extensions) feature enables efficient tile-based computation for deep learning models, especially for INT8 precision operations. By dividing large models into smaller 16×16 blocks, AMX-TILE accelerates matrix operations, reducing latency and optimizing memory usage. This results in faster processing times for AI inference tasks, without the need for additional hardware like GPUs.

• VNNI (Vector Neural Network Instructions)

VNNI enhances vector processing by supporting operations such as multiply-accumulate (MAC), a critical operation for neural networks. It significantly improves the data throughput and efficiency of AI inference, allowing larger models to be processed more efficiently on CPU-based systems. VNNI boosts the computational power of CPUs, allowing them to handle AI workloads typically reserved for GPUs.