{"id": 1015512, "name": "Training computation (petaFLOP)", "unit": "", "createdAt": "2025-03-15T08:53:18.000Z", "updatedAt": "2026-03-08T06:31:58.000Z", "coverage": "", "timespan": "", "datasetId": 6999, "shortUnit": "", "columnOrder": 0, "shortName": "training_computation_petaflop", "catalogPath": "grapher/artificial_intelligence/2025-03-12/epoch/epoch#training_computation_petaflop", "descriptionShort": "Computation is measured in total petaFLOP, which is 10\u00b9\u2075 [floating-point operations](#dod:flop) estimated from AI literature, albeit with some uncertainty.", "type": "float", "grapherConfigIdETL": "01959901-f8eb-77c1-acea-ebd758bb81a7", "dataChecksum": "1707529007208644087", "metadataChecksum": "7185371890693072136", "datasetName": "Parameter, Compute and Data Trends in Machine Learning", "updatePeriodDays": 31, "datasetVersion": "2025-03-12", "nonRedistributable": false, "display": {"zeroDay": "1949-01-01", "yearIsDay": true}, "schemaVersion": 2, "processingLevel": "major", "presentation": {"topicTagsLinks": ["Artificial Intelligence"]}, "descriptionKey": ["In the context of artificial intelligence (AI), training computation is predominantly measured using floating-point operations or \u201cFLOP\u201d. One FLOP represents a single arithmetic operation involving floating-point numbers, such as addition, subtraction, multiplication, or division. To adapt to the vast computational demands of AI systems, the measurement unit of petaFLOP is commonly used. One petaFLOP stands as a staggering one quadrillion FLOPs, underscoring the magnitude of computational operations within AI.", "Modern AI systems are rooted in machine learning and deep learning techniques. These methodologies are notorious for their computational intensity, involving complex mathematical processes and algorithms. During the training phase, AI models process large volumes of data, while continuously adapting and refining their parameters to optimize performance, rendering the training process computationally intensive.", "Many factors influence the magnitude of training computation within AI systems. Notably, the size of the dataset employed for training significantly impacts the computational load. Larger datasets necessitate more processing power. The complexity of the model's architecture also plays a pivotal role; more intricate models lead to more computations. Parallel processing, involving the simultaneous use of multiple processors, also has a substantial effect. Beyond these factors, specific design choices and other variables further contribute to the complexity and scale of training computation within AI."], "dimensions": {"years": {"values": [{"id": 25443}, {"id": 25282}, {"id": 23936}, {"id": 4198}, {"id": 24096}, {"id": 26428}, {"id": 27603}, {"id": 25971}, {"id": 26459}, {"id": 27534}, {"id": 16403}, {"id": 26986}, {"id": 12661}, {"id": 25727}, {"id": 25055}, {"id": 25105}, {"id": 25700}, {"id": 25150}, {"id": 26476}, {"id": 28017}, {"id": 23283}, {"id": 26876}, {"id": 26695}, {"id": 25946}, {"id": 26266}, {"id": 27521}, {"id": 26574}, {"id": 24379}, {"id": 24497}, {"id": 25128}, {"id": 25127}, {"id": 25869}, {"id": 26940}, {"id": 25841}, {"id": 25175}, {"id": 27730}, {"id": 27298}, {"id": 27043}, {"id": 27456}, {"id": 27091}, {"id": 27234}, {"id": 26620}, {"id": 26896}, {"id": 25485}, {"id": 25676}, {"id": 26759}, {"id": 24780}, {"id": 27057}, {"id": 26854}, {"id": 15142}, {"id": 25871}, {"id": 25441}, {"id": 25392}, {"id": 26884}, {"id": 27116}, {"id": 26445}, {"id": 27681}, {"id": 26302}, {"id": 22905}, {"id": 22920}, {"id": 27326}, {"id": 26267}, {"id": 26291}, {"id": 27015}, {"id": 25880}, {"id": 15994}, {"id": 16442}, {"id": 27327}, {"id": 26750}, {"id": 27219}, {"id": 27564}, {"id": 27813}, {"id": 26457}, {"id": 26827}, {"id": 26602}, {"id": 26848}, {"id": 26485}, {"id": 27375}, {"id": 27337}, {"id": 26443}, {"id": 9739}, {"id": 26225}, {"id": 25950}, {"id": 26647}, {"id": 25042}, {"id": 25660}, {"id": 26758}, {"id": 27479}, {"id": 24324}, {"id": 25919}, {"id": 27054}, {"id": 26078}, {"id": 27131}, {"id": 26819}, {"id": 25717}, {"id": 26337}, {"id": 26524}, {"id": 23347}, {"id": 23728}, {"id": 26458}, {"id": 26147}, {"id": 19334}, {"id": 22763}, {"id": 25024}, {"id": 27561}, {"id": 27778}, {"id": 27906}, {"id": 27642}, {"id": 27751}, {"id": 27841}, {"id": 24842}, {"id": 26312}, {"id": 26289}, {"id": 26669}, {"id": 27088}, {"id": 26939}, {"id": 26994}, {"id": 23391}, {"id": 13740}, {"id": 27694}, {"id": 27037}, {"id": 23164}, {"id": 25324}, {"id": 25062}, {"id": 26014}, {"id": 25233}, {"id": 26483}, {"id": 26654}, {"id": 26297}, {"id": 26150}, {"id": 26281}, {"id": 26864}, {"id": 27569}, {"id": 26980}, {"id": 27736}, {"id": 27954}, {"id": 27833}, {"id": 26471}, {"id": 24828}, {"id": 25976}, {"id": 27921}, {"id": 26543}, {"id": 25385}, {"id": 27276}, {"id": 27101}, {"id": 25983}, {"id": 22412}, {"id": 27307}, {"id": 25517}, {"id": 26781}, {"id": 26955}, {"id": 26623}, {"id": 26472}, {"id": 26715}, {"id": 24092}, {"id": 25140}, {"id": 26984}, {"id": 23901}, {"id": 27244}, {"id": 25077}, {"id": 26878}, {"id": 27975}, {"id": 28031}, {"id": 28114}, {"id": 26644}, {"id": 24740}, {"id": 21892}, {"id": 26505}, {"id": 25353}, {"id": 25611}, {"id": 26809}, {"id": 26080}, {"id": 26736}, {"id": 27816}, {"id": 27977}, {"id": 26702}, {"id": 22080}, {"id": 27384}, {"id": 26113}, {"id": 26982}, {"id": 26794}, {"id": 27367}, {"id": 27514}, {"id": 26946}, {"id": 26361}, {"id": 26226}, {"id": 24000}, {"id": 26639}, {"id": 27345}, {"id": 27806}, {"id": 27948}, {"id": 27335}, {"id": 27618}, {"id": 24818}, {"id": 25598}, {"id": 14940}, {"id": 6117}, {"id": 20459}, {"id": 23588}, {"id": 22841}, {"id": 27703}, {"id": 27828}, {"id": 27024}, {"id": 27205}, {"id": 26550}, {"id": 25237}, {"id": 25094}, {"id": 23729}, {"id": 26805}, {"id": 24441}, {"id": 27126}, {"id": 27353}, {"id": 27459}, {"id": 27158}, {"id": 26689}, {"id": 26976}, {"id": 27214}, {"id": 20266}, {"id": 14778}, {"id": 25027}, {"id": 16771}, {"id": 27268}, {"id": 26520}, {"id": 28123}, {"id": 26259}, {"id": 21356}, {"id": 25624}, {"id": 27950}, {"id": 28068}, {"id": 22240}, {"id": 17131}, {"id": 27082}, {"id": 27134}, {"id": 27336}, {"id": 27611}, {"id": 20423}, {"id": 17850}, {"id": 23262}, {"id": 25468}, {"id": 6513}, {"id": 26207}, {"id": 26703}, {"id": 18201}, {"id": 25067}, {"id": 4899}, {"id": 28041}, {"id": 27226}, {"id": 27501}, {"id": 27597}, {"id": 27660}, {"id": 27733}, {"id": 27853}, {"id": 27368}, {"id": 27556}, {"id": 28002}, {"id": 14457}, {"id": 13787}, {"id": 27466}, {"id": 25905}, {"id": 26341}, {"id": 24142}, {"id": 17320}, {"id": 26745}, {"id": 26612}, {"id": 547}, {"id": 2922}, {"id": 3683}, {"id": 4106}, {"id": 14035}, {"id": 14944}, {"id": 15826}, {"id": 18959}, {"id": 24077}, {"id": 25323}, {"id": 26842}, {"id": 11413}, {"id": 26308}, {"id": 26437}, {"id": 25959}, {"id": 27446}, {"id": 25826}, {"id": 26581}, {"id": 25510}, {"id": 26015}, {"id": 26357}, {"id": 27449}, {"id": 27598}, {"id": 27372}, {"id": 24859}, {"id": 26969}, {"id": 27670}, {"id": 25889}, {"id": 25520}, {"id": 27339}, {"id": 25681}, {"id": 15126}, {"id": 26849}, {"id": 19796}, {"id": 27688}, {"id": 24534}, {"id": 27346}, {"id": 27558}, {"id": 25881}, {"id": 27042}, {"id": 26625}, {"id": 27165}, {"id": 26784}, {"id": 27533}, {"id": 28082}, {"id": 26865}, {"id": 26051}, {"id": 25913}, {"id": 25059}, {"id": 27557}, {"id": 26560}, {"id": 26406}, {"id": 26919}, {"id": 27176}, {"id": 26756}, {"id": 27157}, {"id": 27106}, {"id": 27858}, {"id": 27213}, {"id": 26835}, {"id": 27267}, {"id": 26546}, {"id": 25758}, {"id": 26719}, {"id": 24792}, {"id": 22537}, {"id": 5113}, {"id": 26176}, {"id": 26840}, {"id": 26421}, {"id": 25085}, {"id": 27823}, {"id": 27361}, {"id": 27427}, {"id": 27551}, {"id": 27655}, {"id": 27653}, {"id": 27877}, {"id": 27964}, {"id": 27961}, {"id": 28006}, {"id": 28023}, {"id": 24643}, {"id": 23649}, {"id": 27676}, {"id": 18414}, {"id": 26700}, {"id": 22548}, {"id": 23984}, {"id": 20672}, {"id": 23521}, {"id": 27498}, {"id": 24791}, {"id": 24449}, {"id": 24731}, {"id": 25748}, {"id": 27309}, {"id": 26601}, {"id": 24406}, {"id": 26507}, {"id": 17562}, {"id": 27344}, {"id": 24772}, {"id": 23997}, {"id": 23909}, {"id": 26352}, {"id": 26710}, {"id": 20986}, {"id": 3833}, {"id": 25651}, {"id": 27889}, {"id": 26742}, {"id": 27122}, {"id": 23993}, {"id": 15248}, {"id": 16283}, {"id": 27113}, {"id": 27330}, {"id": 23922}, {"id": 26766}, {"id": 26765}, {"id": 27445}, {"id": 27156}, {"id": 26723}, {"id": 26637}, {"id": 25547}, {"id": 26469}, {"id": 27269}, {"id": 26619}, {"id": 17335}, {"id": 25862}, {"id": 24073}, {"id": 24201}, {"id": 25970}, {"id": 27656}, {"id": 26008}, {"id": 23714}, {"id": 24999}, {"id": 25472}, {"id": 25461}, {"id": 25575}, {"id": 25897}, {"id": 25721}, {"id": 26002}, {"id": 14044}, {"id": 25489}, {"id": 26568}, {"id": 25975}, {"id": 22158}, {"id": 24243}, {"id": 25813}, {"id": 26792}, {"id": 26054}, {"id": 23203}, {"id": 27032}, {"id": 24779}, {"id": 23987}, {"id": 27373}, {"id": 27516}, {"id": 24455}, {"id": 27000}, {"id": 27068}, {"id": 26731}, {"id": 26456}, {"id": 26227}, {"id": 27115}, {"id": 23691}, {"id": 25664}, {"id": 15675}, {"id": 26926}, {"id": 23664}, {"id": 26651}, {"id": 25875}, {"id": 26526}, {"id": 25718}, {"id": 24751}, {"id": 26515}, {"id": 25299}, {"id": 27333}, {"id": 27526}, {"id": 27684}, {"id": 26582}, {"id": 25343}, {"id": 26587}, {"id": 26968}, {"id": 24181}, {"id": 22443}, {"id": 27382}, {"id": 25800}]}, "entities": {"values": [{"id": 368101, "name": "(ensemble): AWD-LSTM-DOC (fin) \u00d7 5 (WT2)", "code": null}, {"id": 368102, "name": "4 layer QRNN (h=2500)", "code": null}, {"id": 371825, "name": "ACF-WIDER", "code": null}, {"id": 256995, "name": "ADALINE", "code": null}, {"id": 257024, "name": "ADAM (CIFAR-10)", "code": null}, {"id": 368328, "name": "ADM", "code": null}, {"id": 370093, "name": "AFM-on-device", "code": null}, {"id": 370105, "name": "AFM-server", "code": null}, {"id": 257107, "name": "ALBERT-xxlarge", "code": null}, {"id": 257103, "name": "ALIGN", "code": null}, {"id": 371545, "name": "ALLaM\u00a0adapted 70B", "code": null}, {"id": 369966, "name": "ANN Eye Tracker", "code": null}, {"id": 368082, "name": "AR-LDM", "code": null}, {"id": 305984, "name": "ASE+ACE", "code": null}, {"id": 368026, "name": "AWD-LSTM + MoS + Partial Shuffled", "code": null}, {"id": 368011, "name": "AWD-LSTM - 3-layer LSTM (tied) + continuous cache pointer (WT2)", "code": null}, {"id": 368044, "name": "AWD-LSTM+WT+Cache+IOG (WT2)", "code": null}, {"id": 368063, "name": "AWD-LSTM-DRILL + dynamic evaluation\u2020 (WT2)", "code": null}, {"id": 368041, "name": "AWD-LSTM-MoS + dynamic evaluation (WT2, 2017)", "code": null}, {"id": 368080, "name": "Adaptive Input Transformer + RD", "code": null}, {"id": 372330, "name": "AgentFounder-30B", "code": null}, {"id": 240132, "name": "AlexNet", "code": null}, {"id": 354863, "name": "AlexaTM 20B", "code": null}, {"id": 306182, "name": "AlphaCode", "code": null}, {"id": 257063, "name": "AlphaFold", "code": null}, {"id": 368138, "name": "AlphaFold 2", "code": null}, {"id": 371962, "name": "AlphaFold 3", "code": null}, {"id": 368719, "name": "AlphaFold-Multimer", "code": null}, {"id": 257027, "name": "AlphaGo Fan", "code": null}, {"id": 257029, "name": "AlphaGo Lee", "code": null}, {"id": 257033, "name": "AlphaGo Master", "code": null}, {"id": 257039, "name": "AlphaGo Zero", "code": null}, {"id": 257060, "name": "AlphaStar", "code": null}, {"id": 371829, "name": "AlphaTensor", "code": null}, {"id": 257056, "name": "AlphaX-1", "code": null}, {"id": 240145, "name": "AlphaZero", "code": null}, {"id": 371249, "name": "Amazon Nova Pro", "code": null}, {"id": 370131, "name": "Amazon Titan", "code": null}, {"id": 368732, "name": "Ankh_large", "code": null}, {"id": 371244, "name": "Aramco Metabrain AI", "code": null}, {"id": 369016, "name": "AudioGen", "code": null}, {"id": 369002, "name": "AudioLM", "code": null}, {"id": 368364, "name": "BASIC-L", "code": null}, {"id": 369347, "name": "BEIT-3", "code": null}, {"id": 257045, "name": "BERT-Large", "code": null}, {"id": 368077, "name": "BERT-Large-CAS (PTB+WT2+WT103)", "code": null}, {"id": 368730, "name": "BERT-RBP", "code": null}, {"id": 369179, "name": "BIDAF", "code": null}, {"id": 369196, "name": "BLIP-2 (Q-Former)", "code": null}, {"id": 368746, "name": "BLOOM-176B", "code": null}, {"id": 369990, "name": "Bankruptcy-NN", "code": null}, {"id": 368067, "name": "Base LM + kNN LM + Continuous Cache", "code": null}, {"id": 369348, "name": "Big Transformer for Back-Translation", "code": null}, {"id": 368035, "name": "Big-Little Net", "code": null}, {"id": 369018, "name": "Big-Little Net (speech)", "code": null}, {"id": 368716, "name": "BlenderBot 3", "code": null}, {"id": 367081, "name": "BloombergGPT", "code": null}, {"id": 368326, "name": "ByT5-XXL", "code": null}, {"id": 370972, "name": "CHAI-1", "code": null}, {"id": 257076, "name": "CLIP (ViT L/14@336px)", "code": null}, {"id": 371842, "name": "CNN Committee (MNIST)", "code": null}, {"id": 371862, "name": "CNN Committee (NIST)", "code": null}, {"id": 371863, "name": "CNN committee (traffic sign)", "code": null}, {"id": 368129, "name": "CODEFUSION (Python)", "code": null}, {"id": 257074, "name": "CPM-Large", "code": null}, {"id": 368078, "name": "CT-MoS (WT2)", "code": null}, {"id": 368726, "name": "CaLM", "code": null}, {"id": 369328, "name": "CamemBERT", "code": null}, {"id": 369525, "name": "Cancer drug mechanism prediction", "code": null}, {"id": 369972, "name": "Ceramic-MLP", "code": null}, {"id": 369998, "name": "ChatGLM3-6B", "code": null}, {"id": 273166, "name": "Chinchilla", "code": null}, {"id": 367637, "name": "Claude 2", "code": null}, {"id": 370244, "name": "Claude 3.5 Sonnet", "code": null}, {"id": 371367, "name": "Claude 3.7 Sonnet", "code": null}, {"id": 368362, "name": "CoAtNet", "code": null}, {"id": 368373, "name": "CoCa", "code": null}, {"id": 368135, "name": "CodeT5-base", "code": null}, {"id": 368133, "name": "CodeT5-large", "code": null}, {"id": 306167, "name": "Codex", "code": null}, {"id": 369518, "name": "CogAgent", "code": null}, {"id": 370174, "name": "CogVLM-17B", "code": null}, {"id": 257084, "name": "CogView", "code": null}, {"id": 305980, "name": "Cognitron", "code": null}, {"id": 369190, "name": "Conformer + Wav2vec 2.0 + Noisy Student", "code": null}, {"id": 368366, "name": "ContextNet + Noisy Student", "code": null}, {"id": 368725, "name": "Contriever", "code": null}, {"id": 369558, "name": "ConvS2S (ensemble of 8 models)", "code": null}, {"id": 268375, "name": "Cross-lingual alignment", "code": null}, {"id": 257077, "name": "DALL-E", "code": null}, {"id": 306186, "name": "DALL\u00b7E 2", "code": null}, {"id": 369883, "name": "DBRX", "code": null}, {"id": 371859, "name": "DCNN", "code": null}, {"id": 368324, "name": "DD-PPO", "code": null}, {"id": 368354, "name": "DDPM-IP (CelebA)", "code": null}, {"id": 368341, "name": "DETR", "code": null}, {"id": 369173, "name": "DINOv2", "code": null}, {"id": 368096, "name": "DITTO", "code": null}, {"id": 257051, "name": "DLRM-2020", "code": null}, {"id": 371972, "name": "DLWP", "code": null}, {"id": 368748, "name": "DNABERT", "code": null}, {"id": 371854, "name": "DNN EM segmentation", "code": null}, {"id": 240135, "name": "DQN", "code": null}, {"id": 306150, "name": "DeBERTa", "code": null}, {"id": 370248, "name": "DeLighT", "code": null}, {"id": 257009, "name": "Decision tree (classification)", "code": null}, {"id": 370720, "name": "Deep Autoencoders", "code": null}, {"id": 371976, "name": "DeepLoc", "code": null}, {"id": 370239, "name": "DeepSeek-Coder-V2 236B", "code": null}, {"id": 371370, "name": "DeepSeek-R1", "code": null}, {"id": 372346, "name": "DeepSeek-R1 (May 2025)", "code": null}, {"id": 370561, "name": "DeepSeek-V2.5", "code": null}, {"id": 371328, "name": "DeepSeek-V3", "code": null}, {"id": 372634, "name": "DeepSeek-V3 (Mar 2025)", "code": null}, {"id": 257034, "name": "DeepStack", "code": null}, {"id": 366051, "name": "DeiT-B", "code": null}, {"id": 306166, "name": "Denoising Diffusion Probabilistic Models (LSUN Bedroom)", "code": null}, {"id": 369162, "name": "DensePhrases", "code": null}, {"id": 369202, "name": "Detic", "code": null}, {"id": 368750, "name": "DiT-XL/2", "code": null}, {"id": 369185, "name": "DiffDock", "code": null}, {"id": 368749, "name": "Discriminator Guidance", "code": null}, {"id": 369539, "name": "DistBelief NNLM", "code": null}, {"id": 369538, "name": "DistBelief Speech", "code": null}, {"id": 306126, "name": "DistilBERT", "code": null}, {"id": 369560, "name": "Distributed representation NN", "code": null}, {"id": 371248, "name": "Doubao-pro", "code": null}, {"id": 371511, "name": "DreamerV3", "code": null}, {"id": 306036, "name": "Dropout (CIFAR)", "code": null}, {"id": 306037, "name": "Dropout (ImageNet)", "code": null}, {"id": 257017, "name": "Dropout (MNIST)", "code": null}, {"id": 368004, "name": "Dropout-LSTM+Noise(Bernoulli) (WT2)", "code": null}, {"id": 368056, "name": "EI-REHN-1000D", "code": null}, {"id": 306136, "name": "ELECTRA", "code": null}, {"id": 306099, "name": "ELMo", "code": null}, {"id": 368718, "name": "EMDR", "code": null}, {"id": 257087, "name": "ERNIE 3.0", "code": null}, {"id": 368055, "name": "ERNIE 3.0 Titan", "code": null}, {"id": 368012, "name": "ERNIE-Doc (247M)", "code": null}, {"id": 306133, "name": "ERNIE-GEN (large)", "code": null}, {"id": 369335, "name": "ESM1b", "code": null}, {"id": 369022, "name": "ESM2-15B", "code": null}, {"id": 369981, "name": "ESM3 (98B)", "code": null}, {"id": 369545, "name": "EVA-01", "code": null}, {"id": 371364, "name": "EXAONE 3.5 32B", "code": null}, {"id": 371817, "name": "EXAONE 4.0 (32B)", "code": null}, {"id": 371472, "name": "EXAONE Deep 32B", "code": null}, {"id": 371654, "name": "Eagle 2", "code": null}, {"id": 370177, "name": "EfficientNetV2-XL", "code": null}, {"id": 371866, "name": "EnhanceNet", "code": null}, {"id": 372320, "name": "FFN SwiGLU", "code": null}, {"id": 371850, "name": "FGN", "code": null}, {"id": 368372, "name": "FLAN 137B", "code": null}, {"id": 371983, "name": "FTW (For The Win)", "code": null}, {"id": 369508, "name": "Falcon-180B", "code": null}, {"id": 367636, "name": "Falcon-40B", "code": null}, {"id": 368047, "name": "Feedback Transformer", "code": null}, {"id": 257013, "name": "Feedforward NN", "code": null}, {"id": 368729, "name": "FinGPT-13B", "code": null}, {"id": 370722, "name": "Fine-tuned-AWD-LSTM-DOC (fin)", "code": null}, {"id": 306187, "name": "Flamingo", "code": null}, {"id": 369171, "name": "Flan-PaLM 540B", "code": null}, {"id": 35176, "name": "Florence", "code": null}, {"id": 369017, "name": "Fold2Seq", "code": null}, {"id": 371965, "name": "FourCastNet", "code": null}, {"id": 368346, "name": "Fractional Max-Pooling", "code": null}, {"id": 368045, "name": "Fraternal dropout + AWD-LSTM 3-layer (WT2)", "code": null}, {"id": 369532, "name": "FunSearch", "code": null}, {"id": 368735, "name": "Fusion in Encoder", "code": null}, {"id": 257021, "name": "GANs", "code": null}, {"id": 369035, "name": "GGNN", "code": null}, {"id": 368086, "name": "GL-LWGC-AWD-MoS-LSTM + dynamic evaluation (WT2)", "code": null}, {"id": 365992, "name": "GLM-130B", "code": null}, {"id": 372366, "name": "GLM-4.5", "code": null}, {"id": 372365, "name": "GLM-4.6", "code": null}, {"id": 372369, "name": "GLM-4.7", "code": null}, {"id": 368065, "name": "GLaM", "code": null}, {"id": 257030, "name": "GNMT", "code": null}, {"id": 371852, "name": "GNN", "code": null}, {"id": 273165, "name": "GOAT", "code": null}, {"id": 370176, "name": "GPT-1", "code": null}, {"id": 369043, "name": "GPT-2 (1.5B)", "code": null}, {"id": 371857, "name": "GPT-2 Medium (FlashAttention)", "code": null}, {"id": 354864, "name": "GPT-3 175B (davinci)", "code": null}, {"id": 372371, "name": "GPT-3.5 (davinci-002)\n", "code": null}, {"id": 372333, "name": "GPT-4 (Jun 2023)", "code": null}, {"id": 372308, "name": "GPT-4 (Mar 2023)", "code": null}, {"id": 371369, "name": "GPT-4.5", "code": null}, {"id": 371945, "name": "GPT-5", "code": null}, {"id": 257096, "name": "GPT-NeoX-20B", "code": null}, {"id": 257011, "name": "GPU DBNs", "code": null}, {"id": 372331, "name": "GQA-8-XXL", "code": null}, {"id": 257072, "name": "GShard (dense)", "code": null}, {"id": 368109, "name": "Galactica", "code": null}, {"id": 306191, "name": "Gato", "code": null}, {"id": 369506, "name": "Gemini 1.0 Ultra", "code": null}, {"id": 371961, "name": "GenCast", "code": null}, {"id": 369025, "name": "GenSLM", "code": null}, {"id": 307047, "name": "Generative BST", "code": null}, {"id": 369146, "name": "German ELECTRA Large", "code": null}, {"id": 257026, "name": "GoogLeNet / InceptionV1", "code": null}, {"id": 367255, "name": "Gopher (280B)", "code": null}, {"id": 369547, "name": "GraphCast", "code": null}, {"id": 371865, "name": "Grok 3", "code": null}, {"id": 371820, "name": "Grok 4", "code": null}, {"id": 368376, "name": "Grok-1", "code": null}, {"id": 371231, "name": "Grok-2", "code": null}, {"id": 371890, "name": "HR-ResNet101", "code": null}, {"id": 257046, "name": "Hanabi 4 player", "code": null}, {"id": 372312, "name": "Handwritten digit recognition network", "code": null}, {"id": 369531, "name": "Heuristic Reinforcement Learning", "code": null}, {"id": 369987, "name": "Hierarchical LM", "code": null}, {"id": 371980, "name": "Hierarchical Scene Labeling (Stanford Background)", "code": null}, {"id": 371885, "name": "High Performance CNN (NORB)", "code": null}, {"id": 257088, "name": "HuBERT", "code": null}, {"id": 371242, "name": "Hunyuan-Large", "code": null}, {"id": 371513, "name": "Hunyuan-TurboS", "code": null}, {"id": 368002, "name": "Hybrid H3-2.7B", "code": null}, {"id": 369030, "name": "HyenaDNA", "code": null}, {"id": 369563, "name": "HyperCLOVA 204B", "code": null}, {"id": 257040, "name": "IMPALA", "code": null}, {"id": 368023, "name": "ISS", "code": null}, {"id": 306044, "name": "Image generation", "code": null}, {"id": 306192, "name": "Imagen", "code": null}, {"id": 306064, "name": "Inception v3", "code": null}, {"id": 368359, "name": "Incoder-6.7B", "code": null}, {"id": 368751, "name": "Inflection-2", "code": null}, {"id": 369561, "name": "Inflection-2.5", "code": null}, {"id": 369334, "name": "InstructBLIP", "code": null}, {"id": 371237, "name": "InstructGPT 175B", "code": null}, {"id": 369198, "name": "InternImage", "code": null}, {"id": 367509, "name": "InternLM", "code": null}, {"id": 369971, "name": "Invariant CNN", "code": null}, {"id": 369549, "name": "Invariant image recognition", "code": null}, {"id": 257037, "name": "JFT", "code": null}, {"id": 369964, "name": "JPMAX", "code": null}, {"id": 367533, "name": "Jais", "code": null}, {"id": 257090, "name": "Jurassic-1-Jumbo", "code": null}, {"id": 372363, "name": "K-EXAONE", "code": null}, {"id": 268376, "name": "KEPLER", "code": null}, {"id": 369975, "name": "KN-LM", "code": null}, {"id": 366991, "name": "KataGo", "code": null}, {"id": 371815, "name": "Kimi K2", "code": null}, {"id": 372341, "name": "Kimi K2 Thinking", "code": null}, {"id": 371881, "name": "LCNP LabelMe", "code": null}, {"id": 371878, "name": "LCNP MNIST", "code": null}, {"id": 371874, "name": "LCNP NORB", "code": null}, {"id": 369970, "name": "LISSOM", "code": null}, {"id": 367499, "name": "LLaMA-65B", "code": null}, {"id": 369186, "name": "LLaVA", "code": null}, {"id": 369331, "name": "LLaVA 1.5", "code": null}, {"id": 371536, "name": "LLaVA-OV-72B", "code": null}, {"id": 369988, "name": "LMICA", "code": null}, {"id": 256998, "name": "LSTM", "code": null}, {"id": 369534, "name": "LSTM LM", "code": null}, {"id": 368009, "name": "LSTM+NeuralCache", "code": null}, {"id": 369517, "name": "LTE speaker verification system", "code": null}, {"id": 369195, "name": "LUKE", "code": null}, {"id": 257095, "name": "LaMDA", "code": null}, {"id": 245542, "name": "LeNet-5", "code": null}, {"id": 257032, "name": "Libratus", "code": null}, {"id": 369543, "name": "Linear Decision Functions", "code": null}, {"id": 372318, "name": "Ling-1T", "code": null}, {"id": 368743, "name": "Llama 2-70B", "code": null}, {"id": 368747, "name": "Llama 2-7B", "code": null}, {"id": 369516, "name": "Llama 3-70B", "code": null}, {"id": 370155, "name": "Llama 3.1-405B", "code": null}, {"id": 371540, "name": "Llama 3.2 11B", "code": null}, {"id": 371535, "name": "Llama 3.3 70B", "code": null}, {"id": 371514, "name": "Llama 4 Behemoth (preview)", "code": null}, {"id": 371860, "name": "Llama 4 Maverick", "code": null}, {"id": 371843, "name": "Llama 4 Scout", "code": null}, {"id": 369174, "name": "Llama Guard", "code": null}, {"id": 371839, "name": "Llama-3.1-Nemotron-70B-Instruct", "code": null}, {"id": 372167, "name": "LongCat-Flash", "code": null}, {"id": 306163, "name": "M6-T", "code": null}, {"id": 369973, "name": "MLN-ASR", "code": null}, {"id": 372328, "name": "MLP with back-propagation", "code": null}, {"id": 369524, "name": "MM1-30B", "code": null}, {"id": 371947, "name": "MMLSTM (PTB)", "code": null}, {"id": 371951, "name": "MMLSTM (WT-2)", "code": null}, {"id": 368753, "name": "MSA Transformer", "code": null}, {"id": 257025, "name": "MSRA (C, PReLU)", "code": null}, {"id": 369993, "name": "MUSIC perceptron", "code": null}, {"id": 371963, "name": "Make-A-Scene", "code": null}, {"id": 370718, "name": "Masked Autoencoders ViT-H", "code": null}, {"id": 370172, "name": "Maximum compute", "code": null}, {"id": 370173, "name": "Maximum data", "code": null}, {"id": 370175, "name": "Maximum parameters", "code": null}, {"id": 369184, "name": "MedBERT", "code": null}, {"id": 257064, "name": "Meena", "code": null}, {"id": 369522, "name": "MegaScale (Production)", "code": null}, {"id": 257055, "name": "Megatron-BERT", "code": null}, {"id": 371869, "name": "Megatron-LM (1.2B)", "code": null}, {"id": 368027, "name": "Megatron-LM (8.3B)", "code": null}, {"id": 257092, "name": "Megatron-Turing NLG 530B", "code": null}, {"id": 369326, "name": "Mesh-TensorFlow Transformer 2.9B (translation)", "code": null}, {"id": 370525, "name": "Mesh-TensorFlow Transformer 4.9B (language)", "code": null}, {"id": 306137, "name": "MetNet", "code": null}, {"id": 257104, "name": "Meta Pseudo Labels", "code": null}, {"id": 306195, "name": "Minerva (540B)", "code": null}, {"id": 369880, "name": "Mistral Large", "code": null}, {"id": 370085, "name": "Mistral Large 2", "code": null}, {"id": 369027, "name": "Mixtral 8x7B", "code": null}, {"id": 369991, "name": "Mixture of linear models", "code": null}, {"id": 370178, "name": "MoE-Multi", "code": null}, {"id": 368006, "name": "Mogrifier RLSTM (WT2)", "code": null}, {"id": 371329, "name": "Movie Gen Video", "code": null}, {"id": 306130, "name": "MuZero", "code": null}, {"id": 368028, "name": "Multi-cell LSTM", "code": null}, {"id": 369014, "name": "MultiBand Diffusion", "code": null}, {"id": 371848, "name": "MuseNet", "code": null}, {"id": 370527, "name": "NAS with base 8 and shared embeddings", "code": null}, {"id": 257031, "name": "NASv3 (CIFAR-10)", "code": null}, {"id": 371841, "name": "NETtalk reimplementation", "code": null}, {"id": 306196, "name": "NLLB", "code": null}, {"id": 371812, "name": "NPLM (AP News)", "code": null}, {"id": 371816, "name": "NPLM (Brown)", "code": null}, {"id": 371240, "name": "NVLM-D 72B", "code": null}, {"id": 371234, "name": "NVLM-H 72B", "code": null}, {"id": 371232, "name": "NVLM-X 72B", "code": null}, {"id": 257109, "name": "Named Entity Recognition model", "code": null}, {"id": 369019, "name": "Nemotron-3-8B", "code": null}, {"id": 369982, "name": "Nemotron-4 340B", "code": null}, {"id": 256996, "name": "Neocognitron", "code": null}, {"id": 368075, "name": "NetTalk (dictionary)", "code": null}, {"id": 368083, "name": "NetTalk (transcription)", "code": null}, {"id": 369997, "name": "Neural LM", "code": null}, {"id": 369992, "name": "NeuroChess", "code": null}, {"id": 306128, "name": "Noisy Student (L2)", "code": null}, {"id": 369009, "name": "Nucleotide Transformer", "code": null}, {"id": 306174, "name": "N\u00dcWA", "code": null}, {"id": 366449, "name": "ONE-PEACE", "code": null}, {"id": 306188, "name": "OPT-175B", "code": null}, {"id": 371524, "name": "Octo-Base", "code": null}, {"id": 372433, "name": "Olmo 3", "code": null}, {"id": 369033, "name": "OmegaPLM", "code": null}, {"id": 257068, "name": "Once for All", "code": null}, {"id": 257061, "name": "OpenAI Five", "code": null}, {"id": 257062, "name": "OpenAI Five Rerun", "code": null}, {"id": 257038, "name": "OpenAI TI7 DOTA 1v1", "code": null}, {"id": 370246, "name": "OpenVLA", "code": null}, {"id": 366990, "name": "PLATO-XL", "code": null}, {"id": 355353, "name": "PLUG", "code": null}, {"id": 368374, "name": "PaLI", "code": null}, {"id": 368325, "name": "PaLI-X", "code": null}, {"id": 273167, "name": "PaLM (540B)", "code": null}, {"id": 365387, "name": "PaLM 2", "code": null}, {"id": 368069, "name": "PanGu-\u03a3", "code": null}, {"id": 257003, "name": "Pandemonium (morse)", "code": null}, {"id": 371529, "name": "Pangu Ultra", "code": null}, {"id": 369204, "name": "Pangu-Weather", "code": null}, {"id": 306194, "name": "Parti", "code": null}, {"id": 369042, "name": "PeptideBERT", "code": null}, {"id": 369024, "name": "Perceptron (1960)", "code": null}, {"id": 257002, "name": "Perceptron Mark I", "code": null}, {"id": 368104, "name": "PermuteFormer", "code": null}, {"id": 368039, "name": "Pluribus", "code": null}, {"id": 369980, "name": "PoE MNIST", "code": null}, {"id": 368076, "name": "Pointer Sentinel-LSTM (medium)", "code": null}, {"id": 368367, "name": "PolyCoder", "code": null}, {"id": 306078, "name": "PolyNet", "code": null}, {"id": 371871, "name": "Pooling CNN (Caltech 101)", "code": null}, {"id": 371872, "name": "Pooling CNN (NORB)", "code": null}, {"id": 369996, "name": "Predictive Coding NN", "code": null}, {"id": 369509, "name": "Print Recognition Logic", "code": null}, {"id": 368338, "name": "ProBERTa", "code": null}, {"id": 369037, "name": "ProGen2-xlarge", "code": null}, {"id": 368339, "name": "Projected GAN", "code": null}, {"id": 369015, "name": "ProtBERT-BFD", "code": null}, {"id": 371533, "name": "ProtT5-XL-U50", "code": null}, {"id": 368350, "name": "ProteinBERT", "code": null}, {"id": 368342, "name": "PyramidNet", "code": null}, {"id": 368074, "name": "QRNN", "code": null}, {"id": 371365, "name": "QwQ-32B", "code": null}, {"id": 368736, "name": "Qwen-72B", "code": null}, {"id": 371236, "name": "Qwen1.5-72B", "code": null}, {"id": 370106, "name": "Qwen2-72B", "code": null}, {"id": 371475, "name": "Qwen2.5 Instruct (72B)", "code": null}, {"id": 371474, "name": "Qwen2.5-32B", "code": null}, {"id": 370534, "name": "Qwen2.5-72B", "code": null}, {"id": 371991, "name": "Qwen3-235B-A22B", "code": null}, {"id": 372632, "name": "Qwen3-235B-A22B (Jul 2025)", "code": null}, {"id": 372358, "name": "Qwen3-235B-A22B-Thinking (Jul 2025)", "code": null}, {"id": 371987, "name": "Qwen3-Coder-480B-A35B", "code": null}, {"id": 371992, "name": "Qwen3-Max", "code": null}, {"id": 372169, "name": "Qwen3-Omni-30B-A3B", "code": null}, {"id": 257105, "name": "R-FCN", "code": null}, {"id": 369513, "name": "RCTM", "code": null}, {"id": 371653, "name": "RDT-1B", "code": null}, {"id": 371856, "name": "RECONTRA-categorized", "code": null}, {"id": 371849, "name": "RECONTRA-uncategorized", "code": null}, {"id": 306180, "name": "RETRO-7B", "code": null}, {"id": 369528, "name": "RNN LM", "code": null}, {"id": 257022, "name": "RNNsearch-50*", "code": null}, {"id": 369535, "name": "RNTN", "code": null}, {"id": 370522, "name": "RankNet", "code": null}, {"id": 369974, "name": "ReLU-Speech", "code": null}, {"id": 370170, "name": "Reka Core", "code": null}, {"id": 372332, "name": "ResNeXt-101 (64\u00d74d)", "code": null}, {"id": 306104, "name": "ResNeXt-101 32x48d", "code": null}, {"id": 371899, "name": "ResNet-101 (ImageNet)", "code": null}, {"id": 257028, "name": "ResNet-152 (ImageNet)", "code": null}, {"id": 366658, "name": "ResNet-200", "code": null}, {"id": 308274, "name": "RetinaNet-R101", "code": null}, {"id": 365388, "name": "RoBERTa Large", "code": null}, {"id": 372317, "name": "RoFormer", "code": null}, {"id": 371966, "name": "RoseTTAFold All-Atom (RFAA)", "code": null}, {"id": 368092, "name": "S4", "code": null}, {"id": 371882, "name": "SAF R-CNN", "code": null}, {"id": 369976, "name": "SB-LM", "code": null}, {"id": 257089, "name": "SEER", "code": null}, {"id": 370721, "name": "SNM-skip", "code": null}, {"id": 369979, "name": "SOM-CNN", "code": null}, {"id": 369144, "name": "SPHINX (Llama 2 13B)", "code": null}, {"id": 369001, "name": "SPIDER2", "code": null}, {"id": 368042, "name": "SPN-4+KN5", "code": null}, {"id": 257097, "name": "SPPNet", "code": null}, {"id": 368098, "name": "SRU++ Large", "code": null}, {"id": 368348, "name": "ST-MoE", "code": null}, {"id": 371873, "name": "SVM-CNN", "code": null}, {"id": 256994, "name": "Samuel Neural Checkers", "code": null}, {"id": 368005, "name": "Sandwich Transformer", "code": null}, {"id": 369176, "name": "SciBERT", "code": null}, {"id": 371939, "name": "Seed1.5-VL", "code": null}, {"id": 368108, "name": "Segatron-XL large, M=384 + HCP", "code": null}, {"id": 369040, "name": "Segment Anything Model", "code": null}, {"id": 307046, "name": "Seq2Seq LSTM", "code": null}, {"id": 369967, "name": "SexNet compression", "code": null}, {"id": 369969, "name": "Siamese-TDNN", "code": null}, {"id": 371534, "name": "SigLIP 400M", "code": null}, {"id": 368349, "name": "Skywork-13B", "code": null}, {"id": 306046, "name": "SmooCT", "code": null}, {"id": 268378, "name": "Sparse all-MLP", "code": null}, {"id": 369529, "name": "Speaker-independent vowel classification", "code": null}, {"id": 343968, "name": "Stable Diffusion (LDM-KL-8-G)", "code": null}, {"id": 371971, "name": "Stable Diffusion 3", "code": null}, {"id": 368130, "name": "StarCoder", "code": null}, {"id": 306183, "name": "Statement Curriculum Learning", "code": null}, {"id": 369536, "name": "Student of Games", "code": null}, {"id": 371889, "name": "StyleGAN", "code": null}, {"id": 371868, "name": "StyleGAN3-R", "code": null}, {"id": 371879, "name": "StyleGAN3-T", "code": null}, {"id": 367528, "name": "Swift", "code": null}, {"id": 371330, "name": "Swin Transformer V2 (SwinV2-G)", "code": null}, {"id": 257078, "name": "Switch", "code": null}, {"id": 256997, "name": "System 11", "code": null}, {"id": 257059, "name": "T5-11B", "code": null}, {"id": 257058, "name": "T5-3B", "code": null}, {"id": 371867, "name": "TA-CNN", "code": null}, {"id": 371810, "name": "TC-DNN-BLSTM-DNN", "code": null}, {"id": 257007, "name": "TD-Gammon", "code": null}, {"id": 368046, "name": "TaLK Convolution", "code": null}, {"id": 371840, "name": "Telechat2-115B", "code": null}, {"id": 368107, "name": "Tensor-Transformer(1core)+PN (WT103)", "code": null}, {"id": 256993, "name": "Theseus", "code": null}, {"id": 368110, "name": "Tranception", "code": null}, {"id": 257106, "name": "TransE", "code": null}, {"id": 371241, "name": "Transformer (2017)", "code": null}, {"id": 369555, "name": "Transformer (Adaptive Input Embeddings) WT103", "code": null}, {"id": 369007, "name": "Transformer + Simple Recurrent Unit", "code": null}, {"id": 257098, "name": "Transformer local-attention (NesT-B)", "code": null}, {"id": 369511, "name": "Transformer-XL (257M)", "code": null}, {"id": 368010, "name": "Transformer-XL DeFINE (141M)", "code": null}, {"id": 368013, "name": "Transformer-XL Large + Phrase Induction", "code": null}, {"id": 368106, "name": "TransformerXL + spectrum control", "code": null}, {"id": 371864, "name": "Translation-invariant MLP", "code": null}, {"id": 368087, "name": "TrellisNet", "code": null}, {"id": 371948, "name": "Turing ULRv5", "code": null}, {"id": 368060, "name": "Turing-NLG", "code": null}, {"id": 371877, "name": "Two Stage Feature Extraction (MNIST)", "code": null}, {"id": 371876, "name": "U-Net", "code": null}, {"id": 368739, "name": "U-PaLM (540B)", "code": null}, {"id": 369023, "name": "UDSMProt", "code": null}, {"id": 306190, "name": "UL2", "code": null}, {"id": 365990, "name": "UnifiedQA", "code": null}, {"id": 366988, "name": "Unsupervised High-level Feature Learner", "code": null}, {"id": 369034, "name": "VALL-E", "code": null}, {"id": 368057, "name": "VD-LSTM+REAL Large", "code": null}, {"id": 257023, "name": "VGG16", "code": null}, {"id": 306053, "name": "VGG19", "code": null}, {"id": 371814, "name": "VILA-13B", "code": null}, {"id": 371528, "name": "VILA1.5-13B", "code": null}, {"id": 368031, "name": "Variational (untied weights, MC) LSTM (Large)", "code": null}, {"id": 371482, "name": "Vega v2", "code": null}, {"id": 369193, "name": "ViT-22B", "code": null}, {"id": 368361, "name": "ViT-G (model soup)", "code": null}, {"id": 257085, "name": "ViT-G/14", "code": null}, {"id": 306146, "name": "ViT-Huge/14", "code": null}, {"id": 369142, "name": "VideoMAE V2", "code": null}, {"id": 257019, "name": "Visualizing CNNs", "code": null}, {"id": 369349, "name": "Volcano 13B", "code": null}, {"id": 368351, "name": "WeNet (Penn Treebank)", "code": null}, {"id": 369977, "name": "Weight Decay", "code": null}, {"id": 349174, "name": "Whisper", "code": null}, {"id": 257018, "name": "Word2Vec (large)", "code": null}, {"id": 369192, "name": "XGLM-7.5B", "code": null}, {"id": 369168, "name": "XLM-RoBERTa", "code": null}, {"id": 306169, "name": "XLMR-XXL", "code": null}, {"id": 306120, "name": "XLNet", "code": null}, {"id": 240142, "name": "Xception", "code": null}, {"id": 369177, "name": "YOLOX-X", "code": null}, {"id": 257041, "name": "YOLOv3", "code": null}, {"id": 368740, "name": "Yi-34B", "code": null}, {"id": 370141, "name": "Yi-Large", "code": null}, {"id": 371481, "name": "Yi-Lightning", "code": null}, {"id": 257093, "name": "Yuan 1.0", "code": null}, {"id": 367497, "name": "Zidong Taichu", "code": null}, {"id": 257006, "name": "Zip CNN", "code": null}, {"id": 368064, "name": "aLSTM(depth-2)+RecurrentPolicy (WT2)", "code": null}, {"id": 368036, "name": "base LM+GNN+kNN", "code": null}, {"id": 369151, "name": "eDiff-I", "code": null}, {"id": 368084, "name": "genCNN + dyn eval", "code": null}, {"id": 371847, "name": "gpt-oss-120b", "code": null}, {"id": 371855, "name": "gpt-oss-20b", "code": null}, {"id": 371880, "name": "iCCCP", "code": null}, {"id": 368329, "name": "mT5-XXL", "code": null}, {"id": 371483, "name": "nekomata-14b", "code": null}, {"id": 371970, "name": "trRosetta", "code": null}, {"id": 257073, "name": "wave2vec 2.0 LARGE", "code": null}, {"id": 369010, "name": "xTrimoPGLM -100B", "code": null}]}}, "origins": [{"id": 14136, "title": "Parameter, Compute and Data Trends in Machine Learning", "descriptionSnapshot": "We update this chart with the latest available data from our source every month.\n\nThe authors selected the AI systems for inclusion based on the following necessary criteria:\n\u2014 Have an explicit learning component\n\u2014 Showcase experimental results\n\u2014 Advance the state of the art\n\nIn addition, the systems had to meet at least one of the following notability criteria:\n\u2014 Paper has more than 1000 citations\n\u2014 Historical importance\n\u2014 Important state-of-the-art advance\n\u2014 Deployed in a notable context\n\nThe authors note that: \"For new models (from 2020 onward) it is harder to assess these criteria, so we fall back to a subjective selection. We refer to models meeting our selection criteria as 'milestone models.\"\n", "producer": "Epoch AI", "citationFull": "Epoch AI, \u2018Parameter, Compute and Data Trends in Machine Learning\u2019. Published online at epochai.org. Retrieved from: \u2018https://epoch.ai/data/epochdb/visualization\u2019 [online resource]", "urlMain": "https://epoch.ai/mlinputs/visualization", "urlDownload": "https://epoch.ai/data/epochdb/notable_ai_models.csv", "dateAccessed": "2026-03-07", "datePublished": "2025", "license": {"url": "https://creativecommons.org/licenses/by/4.0/", "name": "CC BY 4.0"}}]}