{"id": 1015507, "name": "Training computation (petaFLOP)", "unit": "petaFLOP", "createdAt": "2025-03-15T08:53:18.000Z", "updatedAt": "2026-03-08T06:32:17.000Z", "coverage": "", "timespan": "", "datasetId": 7001, "columnOrder": 0, "shortName": "training_computation_petaflop", "catalogPath": "grapher/artificial_intelligence/2025-03-12/epoch_regressions/epoch_regressions#training_computation_petaflop", "descriptionShort": "Computation is measured in total petaFLOP, which is 10\u00b9\u2075 [floating-point operations](#dod:flop) estimated from AI literature, albeit with some uncertainty.", "type": "float", "grapherConfigIdETL": "01959901-f8a1-7ea2-afc5-1078f906a9b9", "dataChecksum": "1281409811362812417", "metadataChecksum": "-4026915337716187425", "datasetName": "Parameter, Compute and Data Trends in Machine Learning - Regressions", "updatePeriodDays": 31, "datasetVersion": "2025-03-12", "nonRedistributable": false, "display": {"unit": "petaFLOP", "zeroDay": "1949-01-01", "yearIsDay": true, "numDecimalPlaces": 0}, "schemaVersion": 2, "processingLevel": "major", "presentation": {"topicTagsLinks": ["Artificial Intelligence"]}, "descriptionKey": ["In the context of artificial intelligence (AI), training computation is predominantly measured using floating-point operations or \u201cFLOP\u201d. One FLOP represents a single arithmetic operation involving floating-point numbers, such as addition, subtraction, multiplication, or division. To adapt to the vast computational demands of AI systems, the measurement unit of petaFLOP is commonly used. One petaFLOP stands as a staggering one quadrillion FLOPs, underscoring the magnitude of computational operations within AI.", "Modern AI systems are rooted in machine learning and deep learning techniques. These methodologies are notorious for their computational intensity, involving complex mathematical processes and algorithms. During the training phase, AI models process large volumes of data, while continuously adapting and refining their parameters to optimize performance, rendering the training process computationally intensive.", "Many factors influence the magnitude of training computation within AI systems. Notably, the size of the dataset employed for training significantly impacts the computational load. Larger datasets necessitate more processing power. The complexity of the model's architecture also plays a pivotal role; more intricate models lead to more computations. Parallel processing, involving the simultaneous use of multiple processors, also has a substantial effect. Beyond these factors, specific design choices and other variables further contribute to the complexity and scale of training computation within AI."], "dimensions": {"years": {"values": [{"id": 547}, {"id": 2922}, {"id": 3683}, {"id": 3833}, {"id": 4106}, {"id": 4198}, {"id": 4899}, {"id": 5113}, {"id": 6117}, {"id": 6513}, {"id": 9739}, {"id": 11413}, {"id": 12661}, {"id": 13740}, {"id": 13787}, {"id": 14035}, {"id": 14044}, {"id": 14457}, {"id": 14778}, {"id": 14940}, {"id": 14944}, {"id": 15126}, {"id": 15142}, {"id": 15248}, {"id": 15675}, {"id": 15826}, {"id": 15994}, {"id": 16283}, {"id": 16403}, {"id": 16442}, {"id": 16771}, {"id": 17131}, {"id": 17320}, {"id": 17335}, {"id": 17562}, {"id": 17850}, {"id": 18201}, {"id": 18414}, {"id": 18959}, {"id": 19334}, {"id": 19796}, {"id": 20266}, {"id": 20423}, {"id": 20459}, {"id": 20672}, {"id": 20986}, {"id": 21356}, {"id": 21892}, {"id": 22080}, {"id": 22158}, {"id": 22240}, {"id": 22412}, {"id": 22443}, {"id": 22537}, {"id": 22548}, {"id": 22763}, {"id": 22841}, {"id": 22905}, {"id": 22920}, {"id": 23164}, {"id": 23203}, {"id": 23262}, {"id": 23283}, {"id": 23347}, {"id": 23391}, {"id": 23521}, {"id": 23588}, {"id": 23649}, {"id": 23664}, {"id": 23691}, {"id": 23714}, {"id": 23728}, {"id": 23729}, {"id": 23901}, {"id": 23909}, {"id": 23922}, {"id": 23936}, {"id": 23984}, {"id": 23987}, {"id": 23993}, {"id": 23997}, {"id": 24000}, {"id": 24073}, {"id": 24077}, {"id": 24092}, {"id": 24096}, {"id": 24142}, {"id": 24181}, {"id": 24201}, {"id": 24243}, {"id": 24324}, {"id": 24379}, {"id": 24406}, {"id": 24441}, {"id": 24449}, {"id": 24455}, {"id": 24497}, {"id": 24534}, {"id": 24643}, {"id": 24731}, {"id": 24740}, {"id": 24751}, {"id": 24772}, {"id": 24779}, {"id": 24780}, {"id": 24791}, {"id": 24792}, {"id": 24818}, {"id": 24828}, {"id": 24842}, {"id": 24859}, {"id": 24999}, {"id": 25024}, {"id": 25027}, {"id": 25042}, {"id": 25055}, {"id": 25059}, {"id": 25062}, {"id": 25067}, {"id": 25077}, {"id": 25085}, {"id": 25094}, {"id": 25105}, {"id": 25127}, {"id": 25128}, {"id": 25140}, {"id": 25150}, {"id": 25175}, {"id": 25233}, {"id": 25237}, {"id": 25282}, {"id": 25299}, {"id": 25323}, {"id": 25324}, {"id": 25343}, {"id": 25353}, {"id": 25385}, {"id": 25392}, {"id": 25441}, {"id": 25443}, {"id": 25461}, {"id": 25468}, {"id": 25472}, {"id": 25485}, {"id": 25489}, {"id": 25510}, {"id": 25517}, {"id": 25520}, {"id": 25547}, {"id": 25575}, {"id": 25598}, {"id": 25611}, {"id": 25624}, {"id": 25651}, {"id": 25660}, {"id": 25664}, {"id": 25676}, {"id": 25681}, {"id": 25700}, {"id": 25717}, {"id": 25718}, {"id": 25721}, {"id": 25727}, {"id": 25748}, {"id": 25758}, {"id": 25800}, {"id": 25813}, {"id": 25826}, {"id": 25841}, {"id": 25862}, {"id": 25869}, {"id": 25871}, {"id": 25875}, {"id": 25880}, {"id": 25881}, {"id": 25889}, {"id": 25897}, {"id": 25905}, {"id": 25913}, {"id": 25919}, {"id": 25946}, {"id": 25950}, {"id": 25959}, {"id": 25970}, {"id": 25971}, {"id": 25975}, {"id": 25976}, {"id": 25983}, {"id": 26002}, {"id": 26008}, {"id": 26014}, {"id": 26015}, {"id": 26051}, {"id": 26054}, {"id": 26078}, {"id": 26080}, {"id": 26113}, {"id": 26147}, {"id": 26150}, {"id": 26176}, {"id": 26207}, {"id": 26225}, {"id": 26226}, {"id": 26227}, {"id": 26259}, {"id": 26266}, {"id": 26267}, {"id": 26281}, {"id": 26289}, {"id": 26291}, {"id": 26297}, {"id": 26302}, {"id": 26308}, {"id": 26312}, {"id": 26337}, {"id": 26341}, {"id": 26352}, {"id": 26357}, {"id": 26361}, {"id": 26406}, {"id": 26421}, {"id": 26428}, {"id": 26437}, {"id": 26443}, {"id": 26445}, {"id": 26456}, {"id": 26457}, {"id": 26458}, {"id": 26459}, {"id": 26469}, {"id": 26471}, {"id": 26472}, {"id": 26476}, {"id": 26483}, {"id": 26485}, {"id": 26505}, {"id": 26507}, {"id": 26515}, {"id": 26520}, {"id": 26524}, {"id": 26526}, {"id": 26543}, {"id": 26546}, {"id": 26550}, {"id": 26560}, {"id": 26568}, {"id": 26574}, {"id": 26581}, {"id": 26582}, {"id": 26587}, {"id": 26601}, {"id": 26602}, {"id": 26612}, {"id": 26619}, {"id": 26620}, {"id": 26623}, {"id": 26625}, {"id": 26637}, {"id": 26639}, {"id": 26644}, {"id": 26647}, {"id": 26651}, {"id": 26654}, {"id": 26669}, {"id": 26689}, {"id": 26695}, {"id": 26700}, {"id": 26702}, {"id": 26703}, {"id": 26710}, {"id": 26715}, {"id": 26719}, {"id": 26723}, {"id": 26731}, {"id": 26736}, {"id": 26742}, {"id": 26745}, {"id": 26750}, {"id": 26756}, {"id": 26758}, {"id": 26759}, {"id": 26765}, {"id": 26766}, {"id": 26781}, {"id": 26784}, {"id": 26792}, {"id": 26794}, {"id": 26805}, {"id": 26809}, {"id": 26819}, {"id": 26827}, {"id": 26835}, {"id": 26840}, {"id": 26842}, {"id": 26848}, {"id": 26849}, {"id": 26854}, {"id": 26864}, {"id": 26865}, {"id": 26876}, {"id": 26878}, {"id": 26884}, {"id": 26896}, {"id": 26919}, {"id": 26926}, {"id": 26939}, {"id": 26940}, {"id": 26946}, {"id": 26955}, {"id": 26968}, {"id": 26969}, {"id": 26976}, {"id": 26980}, {"id": 26982}, {"id": 26984}, {"id": 26986}, {"id": 26994}, {"id": 27000}, {"id": 27015}, {"id": 27024}, {"id": 27032}, {"id": 27037}, {"id": 27042}, {"id": 27043}, {"id": 27054}, {"id": 27057}, {"id": 27068}, {"id": 27082}, {"id": 27088}, {"id": 27091}, {"id": 27101}, {"id": 27106}, {"id": 27113}, {"id": 27115}, {"id": 27116}, {"id": 27122}, {"id": 27126}, {"id": 27131}, {"id": 27134}, {"id": 27156}, {"id": 27157}, {"id": 27158}, {"id": 27165}, {"id": 27176}, {"id": 27205}, {"id": 27213}, {"id": 27214}, {"id": 27219}, {"id": 27226}, {"id": 27234}, {"id": 27244}, {"id": 27267}, {"id": 27268}, {"id": 27269}, {"id": 27276}, {"id": 27298}, {"id": 27307}, {"id": 27309}, {"id": 27326}, {"id": 27327}, {"id": 27330}, {"id": 27333}, {"id": 27335}, {"id": 27336}, {"id": 27337}, {"id": 27339}, {"id": 27344}, {"id": 27345}, {"id": 27346}, {"id": 27353}, {"id": 27361}, {"id": 27367}, {"id": 27368}, {"id": 27372}, {"id": 27373}, {"id": 27375}, {"id": 27382}, {"id": 27384}, {"id": 27427}, {"id": 27445}, {"id": 27446}, {"id": 27449}, {"id": 27456}, {"id": 27459}, {"id": 27466}, {"id": 27479}, {"id": 27498}, {"id": 27501}, {"id": 27514}, {"id": 27516}, {"id": 27521}, {"id": 27526}, {"id": 27533}, {"id": 27534}, {"id": 27551}, {"id": 27556}, {"id": 27557}, {"id": 27558}, {"id": 27561}, {"id": 27564}, {"id": 27569}, {"id": 27597}, {"id": 27598}, {"id": 27603}, {"id": 27611}, {"id": 27618}, {"id": 27642}, {"id": 27653}, {"id": 27655}, {"id": 27656}, {"id": 27660}, {"id": 27670}, {"id": 27676}, {"id": 27681}, {"id": 27684}, {"id": 27688}, {"id": 27694}, {"id": 27703}, {"id": 27730}, {"id": 27733}, {"id": 27736}, {"id": 27751}, {"id": 27778}, {"id": 27806}, {"id": 27813}, {"id": 27816}, {"id": 27823}, {"id": 27828}, {"id": 27833}, {"id": 27841}, {"id": 27853}, {"id": 27858}, {"id": 27877}, {"id": 27889}, {"id": 27906}, {"id": 27921}, {"id": 27948}, {"id": 27950}, {"id": 27954}, {"id": 27961}, {"id": 27964}, {"id": 27975}, {"id": 27977}, {"id": 28002}, {"id": 28006}, {"id": 28017}, {"id": 28023}, {"id": 28031}, {"id": 28041}, {"id": 28068}, {"id": 28082}, {"id": 28114}, {"id": 28123}]}, "entities": {"values": [{"id": 370902, "name": "1.5x/year between 1950\u20132010", "code": null}, {"id": 256993, "name": "Theseus", "code": null}, {"id": 257002, "name": "Perceptron Mark I", "code": null}, {"id": 257003, "name": "Pandemonium (morse)", "code": null}, {"id": 256994, "name": "Samuel Neural Checkers", "code": null}, {"id": 369024, "name": "Perceptron (1960)", "code": null}, {"id": 256995, "name": "ADALINE", "code": null}, {"id": 369543, "name": "Linear Decision Functions", "code": null}, {"id": 369509, "name": "Print Recognition Logic", "code": null}, {"id": 369531, "name": "Heuristic Reinforcement Learning", "code": null}, {"id": 369517, "name": "LTE speaker verification system", "code": null}, {"id": 305980, "name": "Cognitron", "code": null}, {"id": 256996, "name": "Neocognitron", "code": null}, {"id": 305984, "name": "ASE+ACE", "code": null}, {"id": 369560, "name": "Distributed representation NN", "code": null}, {"id": 372328, "name": "MLP with back-propagation", "code": null}, {"id": 368075, "name": "NetTalk (dictionary)", "code": null}, {"id": 368083, "name": "NetTalk (transcription)", "code": null}, {"id": 371864, "name": "Translation-invariant MLP", "code": null}, {"id": 369973, "name": "MLN-ASR", "code": null}, {"id": 369549, "name": "Invariant image recognition", "code": null}, {"id": 372312, "name": "Handwritten digit recognition network", "code": null}, {"id": 369529, "name": "Speaker-independent vowel classification", "code": null}, {"id": 257006, "name": "Zip CNN", "code": null}, {"id": 371841, "name": "NETtalk reimplementation", "code": null}, {"id": 369990, "name": "Bankruptcy-NN", "code": null}, {"id": 369967, "name": "SexNet compression", "code": null}, {"id": 369977, "name": "Weight Decay", "code": null}, {"id": 257007, "name": "TD-Gammon", "code": null}, {"id": 369525, "name": "Cancer drug mechanism prediction", "code": null}, {"id": 369969, "name": "Siamese-TDNN", "code": null}, {"id": 369966, "name": "ANN Eye Tracker", "code": null}, {"id": 369972, "name": "Ceramic-MLP", "code": null}, {"id": 369964, "name": "JPMAX", "code": null}, {"id": 369991, "name": "Mixture of linear models", "code": null}, {"id": 369992, "name": "NeuroChess", "code": null}, {"id": 369996, "name": "Predictive Coding NN", "code": null}, {"id": 369970, "name": "LISSOM", "code": null}, {"id": 369993, "name": "MUSIC perceptron", "code": null}, {"id": 256997, "name": "System 11", "code": null}, {"id": 369979, "name": "SOM-CNN", "code": null}, {"id": 256998, "name": "LSTM", "code": null}, {"id": 245542, "name": "LeNet-5", "code": null}, {"id": 371856, "name": "RECONTRA-categorized", "code": null}, {"id": 371849, "name": "RECONTRA-uncategorized", "code": null}, {"id": 369997, "name": "Neural LM", "code": null}, {"id": 369980, "name": "PoE MNIST", "code": null}, {"id": 257009, "name": "Decision tree (classification)", "code": null}, {"id": 371812, "name": "NPLM (AP News)", "code": null}, {"id": 371816, "name": "NPLM (Brown)", "code": null}, {"id": 369971, "name": "Invariant CNN", "code": null}, {"id": 369988, "name": "LMICA", "code": null}, {"id": 369987, "name": "Hierarchical LM", "code": null}, {"id": 370522, "name": "RankNet", "code": null}, {"id": 371873, "name": "SVM-CNN", "code": null}, {"id": 369975, "name": "KN-LM", "code": null}, {"id": 369976, "name": "SB-LM", "code": null}, {"id": 371852, "name": "GNN", "code": null}, {"id": 257011, "name": "GPU DBNs", "code": null}, {"id": 371877, "name": "Two Stage Feature Extraction (MNIST)", "code": null}, {"id": 371881, "name": "LCNP LabelMe", "code": null}, {"id": 371878, "name": "LCNP MNIST", "code": null}, {"id": 371874, "name": "LCNP NORB", "code": null}, {"id": 372342, "name": "4.2x/year between 2010\u20132025", "code": null}, {"id": 257013, "name": "Feedforward NN", "code": null}, {"id": 371880, "name": "iCCCP", "code": null}, {"id": 371871, "name": "Pooling CNN (Caltech 101)", "code": null}, {"id": 371872, "name": "Pooling CNN (NORB)", "code": null}, {"id": 369528, "name": "RNN LM", "code": null}, {"id": 370720, "name": "Deep Autoencoders", "code": null}, {"id": 371885, "name": "High Performance CNN (NORB)", "code": null}, {"id": 371842, "name": "CNN Committee (MNIST)", "code": null}, {"id": 371862, "name": "CNN Committee (NIST)", "code": null}, {"id": 371863, "name": "CNN committee (traffic sign)", "code": null}, {"id": 306036, "name": "Dropout (CIFAR)", "code": null}, {"id": 306037, "name": "Dropout (ImageNet)", "code": null}, {"id": 257017, "name": "Dropout (MNIST)", "code": null}, {"id": 366988, "name": "Unsupervised High-level Feature Learner", "code": null}, {"id": 369534, "name": "LSTM LM", "code": null}, {"id": 240132, "name": "AlexNet", "code": null}, {"id": 371854, "name": "DNN EM segmentation", "code": null}, {"id": 369538, "name": "DistBelief Speech", "code": null}, {"id": 369539, "name": "DistBelief NNLM", "code": null}, {"id": 369974, "name": "ReLU-Speech", "code": null}, {"id": 371980, "name": "Hierarchical Scene Labeling (Stanford Background)", "code": null}, {"id": 369513, "name": "RCTM", "code": null}, {"id": 369535, "name": "RNTN", "code": null}, {"id": 257018, "name": "Word2Vec (large)", "code": null}, {"id": 257019, "name": "Visualizing CNNs", "code": null}, {"id": 257106, "name": "TransE", "code": null}, {"id": 240135, "name": "DQN", "code": null}, {"id": 306044, "name": "Image generation", "code": null}, {"id": 257021, "name": "GANs", "code": null}, {"id": 257097, "name": "SPPNet", "code": null}, {"id": 306046, "name": "SmooCT", "code": null}, {"id": 371825, "name": "ACF-WIDER", "code": null}, {"id": 257022, "name": "RNNsearch-50*", "code": null}, {"id": 257023, "name": "VGG16", "code": null}, {"id": 306053, "name": "VGG19", "code": null}, {"id": 307046, "name": "Seq2Seq LSTM", "code": null}, {"id": 368042, "name": "SPN-4+KN5", "code": null}, {"id": 257026, "name": "GoogLeNet / InceptionV1", "code": null}, {"id": 371867, "name": "TA-CNN", "code": null}, {"id": 370721, "name": "SNM-skip", "code": null}, {"id": 368346, "name": "Fractional Max-Pooling", "code": null}, {"id": 257024, "name": "ADAM (CIFAR-10)", "code": null}, {"id": 257025, "name": "MSRA (C, PReLU)", "code": null}, {"id": 368084, "name": "genCNN + dyn eval", "code": null}, {"id": 371810, "name": "TC-DNN-BLSTM-DNN", "code": null}, {"id": 371876, "name": "U-Net", "code": null}, {"id": 371859, "name": "DCNN", "code": null}, {"id": 257027, "name": "AlphaGo Fan", "code": null}, {"id": 371882, "name": "SAF R-CNN", "code": null}, {"id": 306064, "name": "Inception v3", "code": null}, {"id": 371899, "name": "ResNet-101 (ImageNet)", "code": null}, {"id": 257028, "name": "ResNet-152 (ImageNet)", "code": null}, {"id": 368031, "name": "Variational (untied weights, MC) LSTM (Large)", "code": null}, {"id": 257029, "name": "AlphaGo Lee", "code": null}, {"id": 257109, "name": "Named Entity Recognition model", "code": null}, {"id": 257105, "name": "R-FCN", "code": null}, {"id": 366658, "name": "ResNet-200", "code": null}, {"id": 257030, "name": "GNMT", "code": null}, {"id": 368076, "name": "Pointer Sentinel-LSTM (medium)", "code": null}, {"id": 240142, "name": "Xception", "code": null}, {"id": 369001, "name": "SPIDER2", "code": null}, {"id": 368057, "name": "VD-LSTM+REAL Large", "code": null}, {"id": 369179, "name": "BIDAF", "code": null}, {"id": 370527, "name": "NAS with base 8 and shared embeddings", "code": null}, {"id": 257031, "name": "NASv3 (CIFAR-10)", "code": null}, {"id": 372332, "name": "ResNeXt-101 (64\u00d74d)", "code": null}, {"id": 306078, "name": "PolyNet", "code": null}, {"id": 371890, "name": "HR-ResNet101", "code": null}, {"id": 371866, "name": "EnhanceNet", "code": null}, {"id": 257034, "name": "DeepStack", "code": null}, {"id": 370178, "name": "MoE-Multi", "code": null}, {"id": 371241, "name": "Transformer (2017)", "code": null}, {"id": 371976, "name": "DeepLoc", "code": null}, {"id": 257037, "name": "JFT", "code": null}, {"id": 369558, "name": "ConvS2S (ensemble of 8 models)", "code": null}, {"id": 368011, "name": "AWD-LSTM - 3-layer LSTM (tied) + continuous cache pointer (WT2)", "code": null}, {"id": 308274, "name": "RetinaNet-R101", "code": null}, {"id": 257038, "name": "OpenAI TI7 DOTA 1v1", "code": null}, {"id": 368056, "name": "EI-REHN-1000D", "code": null}, {"id": 257032, "name": "Libratus", "code": null}, {"id": 368086, "name": "GL-LWGC-AWD-MoS-LSTM + dynamic evaluation (WT2)", "code": null}, {"id": 368342, "name": "PyramidNet", "code": null}, {"id": 368023, "name": "ISS", "code": null}, {"id": 368044, "name": "AWD-LSTM+WT+Cache+IOG (WT2)", "code": null}, {"id": 257039, "name": "AlphaGo Zero", "code": null}, {"id": 257033, "name": "AlphaGo Master", "code": null}, {"id": 368045, "name": "Fraternal dropout + AWD-LSTM 3-layer (WT2)", "code": null}, {"id": 368041, "name": "AWD-LSTM-MoS + dynamic evaluation (WT2, 2017)", "code": null}, {"id": 240145, "name": "AlphaZero", "code": null}, {"id": 306099, "name": "ELMo", "code": null}, {"id": 368074, "name": "QRNN", "code": null}, {"id": 257040, "name": "IMPALA", "code": null}, {"id": 368102, "name": "4 layer QRNN (h=2500)", "code": null}, {"id": 257041, "name": "YOLOv3", "code": null}, {"id": 306104, "name": "ResNeXt-101 32x48d", "code": null}, {"id": 368004, "name": "Dropout-LSTM+Noise(Bernoulli) (WT2)", "code": null}, {"id": 368064, "name": "aLSTM(depth-2)+RecurrentPolicy (WT2)", "code": null}, {"id": 370176, "name": "GPT-1", "code": null}, {"id": 371983, "name": "FTW (For The Win)", "code": null}, {"id": 368035, "name": "Big-Little Net", "code": null}, {"id": 369018, "name": "Big-Little Net (speech)", "code": null}, {"id": 369348, "name": "Big Transformer for Back-Translation", "code": null}, {"id": 368101, "name": "(ensemble): AWD-LSTM-DOC (fin) \u00d7 5 (WT2)", "code": null}, {"id": 369007, "name": "Transformer + Simple Recurrent Unit", "code": null}, {"id": 368009, "name": "LSTM+NeuralCache", "code": null}, {"id": 369555, "name": "Transformer (Adaptive Input Embeddings) WT103", "code": null}, {"id": 257045, "name": "BERT-Large", "code": null}, {"id": 368087, "name": "TrellisNet", "code": null}, {"id": 369326, "name": "Mesh-TensorFlow Transformer 2.9B (translation)", "code": null}, {"id": 370525, "name": "Mesh-TensorFlow Transformer 4.9B (language)", "code": null}, {"id": 370722, "name": "Fine-tuned-AWD-LSTM-DOC (fin)", "code": null}, {"id": 368028, "name": "Multi-cell LSTM", "code": null}, {"id": 371889, "name": "StyleGAN", "code": null}, {"id": 369511, "name": "Transformer-XL (257M)", "code": null}, {"id": 257046, "name": "Hanabi 4 player", "code": null}, {"id": 369043, "name": "GPT-2 (1.5B)", "code": null}, {"id": 366991, "name": "KataGo", "code": null}, {"id": 369176, "name": "SciBERT", "code": null}, {"id": 268375, "name": "Cross-lingual alignment", "code": null}, {"id": 368351, "name": "WeNet (Penn Treebank)", "code": null}, {"id": 368077, "name": "BERT-Large-CAS (PTB+WT2+WT103)", "code": null}, {"id": 371848, "name": "MuseNet", "code": null}, {"id": 368063, "name": "AWD-LSTM-DRILL + dynamic evaluation\u2020 (WT2)", "code": null}, {"id": 257051, "name": "DLRM-2020", "code": null}, {"id": 306120, "name": "XLNet", "code": null}, {"id": 368013, "name": "Transformer-XL Large + Phrase Induction", "code": null}, {"id": 368026, "name": "AWD-LSTM + MoS + Partial Shuffled", "code": null}, {"id": 365388, "name": "RoBERTa Large", "code": null}, {"id": 368039, "name": "Pluribus", "code": null}, {"id": 371970, "name": "trRosetta", "code": null}, {"id": 369023, "name": "UDSMProt", "code": null}, {"id": 257055, "name": "Megatron-BERT", "code": null}, {"id": 371869, "name": "Megatron-LM (1.2B)", "code": null}, {"id": 368027, "name": "Megatron-LM (8.3B)", "code": null}, {"id": 257056, "name": "AlphaX-1", "code": null}, {"id": 306126, "name": "DistilBERT", "code": null}, {"id": 257059, "name": "T5-11B", "code": null}, {"id": 257058, "name": "T5-3B", "code": null}, {"id": 257060, "name": "AlphaStar", "code": null}, {"id": 368067, "name": "Base LM + kNN LM + Continuous Cache", "code": null}, {"id": 369168, "name": "XLM-RoBERTa", "code": null}, {"id": 369328, "name": "CamemBERT", "code": null}, {"id": 368005, "name": "Sandwich Transformer", "code": null}, {"id": 306128, "name": "Noisy Student (L2)", "code": null}, {"id": 306130, "name": "MuZero", "code": null}, {"id": 368010, "name": "Transformer-XL DeFINE (141M)", "code": null}, {"id": 371947, "name": "MMLSTM (PTB)", "code": null}, {"id": 371951, "name": "MMLSTM (WT-2)", "code": null}, {"id": 257061, "name": "OpenAI Five", "code": null}, {"id": 257062, "name": "OpenAI Five Rerun", "code": null}, {"id": 368324, "name": "DD-PPO", "code": null}, {"id": 257063, "name": "AlphaFold", "code": null}, {"id": 368366, "name": "ContextNet + Noisy Student", "code": null}, {"id": 257064, "name": "Meena", "code": null}, {"id": 368046, "name": "TaLK Convolution", "code": null}, {"id": 257107, "name": "ALBERT-xxlarge", "code": null}, {"id": 368060, "name": "Turing-NLG", "code": null}, {"id": 372320, "name": "FFN SwiGLU", "code": null}, {"id": 368047, "name": "Feedback Transformer", "code": null}, {"id": 368106, "name": "TransformerXL + spectrum control", "code": null}, {"id": 368107, "name": "Tensor-Transformer(1core)+PN (WT103)", "code": null}, {"id": 306136, "name": "ELECTRA", "code": null}, {"id": 306137, "name": "MetNet", "code": null}, {"id": 257068, "name": "Once for All", "code": null}, {"id": 365990, "name": "UnifiedQA", "code": null}, {"id": 368341, "name": "DETR", "code": null}, {"id": 354864, "name": "GPT-3 175B (davinci)", "code": null}, {"id": 257072, "name": "GShard (dense)", "code": null}, {"id": 370248, "name": "DeLighT", "code": null}, {"id": 306133, "name": "ERNIE-GEN (large)", "code": null}, {"id": 368338, "name": "ProBERTa", "code": null}, {"id": 369195, "name": "LUKE", "code": null}, {"id": 369190, "name": "Conformer + Wav2vec 2.0 + Noisy Student", "code": null}, {"id": 368329, "name": "mT5-XXL", "code": null}, {"id": 369146, "name": "German ELECTRA Large", "code": null}, {"id": 306146, "name": "ViT-Huge/14", "code": null}, {"id": 257073, "name": "wave2vec 2.0 LARGE", "code": null}, {"id": 268376, "name": "KEPLER", "code": null}, {"id": 368138, "name": "AlphaFold 2", "code": null}, {"id": 257074, "name": "CPM-Large", "code": null}, {"id": 369335, "name": "ESM1b", "code": null}, {"id": 369162, "name": "DensePhrases", "code": null}, {"id": 368078, "name": "CT-MoS (WT2)", "code": null}, {"id": 368012, "name": "ERNIE-Doc (247M)", "code": null}, {"id": 257076, "name": "CLIP (ViT L/14@336px)", "code": null}, {"id": 257077, "name": "DALL-E", "code": null}, {"id": 257078, "name": "Switch", "code": null}, {"id": 366051, "name": "DeiT-B", "code": null}, {"id": 371972, "name": "DLWP", "code": null}, {"id": 368753, "name": "MSA Transformer", "code": null}, {"id": 368098, "name": "SRU++ Large", "code": null}, {"id": 257104, "name": "Meta Pseudo Labels", "code": null}, {"id": 307047, "name": "Generative BST", "code": null}, {"id": 306163, "name": "M6-T", "code": null}, {"id": 355353, "name": "PLUG", "code": null}, {"id": 369015, "name": "ProtBERT-BFD", "code": null}, {"id": 371533, "name": "ProtT5-XL-U50", "code": null}, {"id": 368328, "name": "ADM", "code": null}, {"id": 369184, "name": "MedBERT", "code": null}, {"id": 257084, "name": "CogView", "code": null}, {"id": 257098, "name": "Transformer local-attention (NesT-B)", "code": null}, {"id": 368326, "name": "ByT5-XXL", "code": null}, {"id": 257085, "name": "ViT-G/14", "code": null}, {"id": 368362, "name": "CoAtNet", "code": null}, {"id": 368718, "name": "EMDR", "code": null}, {"id": 306150, "name": "DeBERTa", "code": null}, {"id": 257103, "name": "ALIGN", "code": null}, {"id": 306166, "name": "Denoising Diffusion Probabilistic Models (LSUN Bedroom)", "code": null}, {"id": 371868, "name": "StyleGAN3-R", "code": null}, {"id": 371879, "name": "StyleGAN3-T", "code": null}, {"id": 370177, "name": "EfficientNetV2-XL", "code": null}, {"id": 369017, "name": "Fold2Seq", "code": null}, {"id": 368080, "name": "Adaptive Input Transformer + RD", "code": null}, {"id": 257087, "name": "ERNIE 3.0", "code": null}, {"id": 306167, "name": "Codex", "code": null}, {"id": 273165, "name": "GOAT", "code": null}, {"id": 257088, "name": "HuBERT", "code": null}, {"id": 257089, "name": "SEER", "code": null}, {"id": 369177, "name": "YOLOX-X", "code": null}, {"id": 257090, "name": "Jurassic-1-Jumbo", "code": null}, {"id": 367497, "name": "Zidong Taichu", "code": null}, {"id": 368748, "name": "DNABERT", "code": null}, {"id": 306169, "name": "XLMR-XXL", "code": null}, {"id": 368372, "name": "FLAN 137B", "code": null}, {"id": 368104, "name": "PermuteFormer", "code": null}, {"id": 369563, "name": "HyperCLOVA 204B", "code": null}, {"id": 366990, "name": "PLATO-XL", "code": null}, {"id": 371948, "name": "Turing ULRv5", "code": null}, {"id": 368719, "name": "AlphaFold-Multimer", "code": null}, {"id": 257092, "name": "Megatron-Turing NLG 530B", "code": null}, {"id": 257093, "name": "Yuan 1.0", "code": null}, {"id": 368036, "name": "base LM+GNN+kNN", "code": null}, {"id": 368092, "name": "S4", "code": null}, {"id": 368135, "name": "CodeT5-base", "code": null}, {"id": 368339, "name": "Projected GAN", "code": null}, {"id": 370718, "name": "Masked Autoencoders ViT-H", "code": null}, {"id": 371330, "name": "Swin Transformer V2 (SwinV2-G)", "code": null}, {"id": 368364, "name": "BASIC-L", "code": null}, {"id": 35176, "name": "Florence", "code": null}, {"id": 306174, "name": "N\u00dcWA", "code": null}, {"id": 369536, "name": "Student of Games", "code": null}, {"id": 367255, "name": "Gopher (280B)", "code": null}, {"id": 368065, "name": "GLaM", "code": null}, {"id": 368725, "name": "Contriever", "code": null}, {"id": 369192, "name": "XGLM-7.5B", "code": null}, {"id": 368055, "name": "ERNIE 3.0 Titan", "code": null}, {"id": 369202, "name": "Detic", "code": null}, {"id": 371237, "name": "InstructGPT 175B", "code": null}, {"id": 306182, "name": "AlphaCode", "code": null}, {"id": 306180, "name": "RETRO-7B", "code": null}, {"id": 257096, "name": "GPT-NeoX-20B", "code": null}, {"id": 257095, "name": "LaMDA", "code": null}, {"id": 368350, "name": "ProteinBERT", "code": null}, {"id": 368348, "name": "ST-MoE", "code": null}, {"id": 371965, "name": "FourCastNet", "code": null}, {"id": 368367, "name": "PolyCoder", "code": null}, {"id": 306183, "name": "Statement Curriculum Learning", "code": null}, {"id": 368361, "name": "ViT-G (model soup)", "code": null}, {"id": 372371, "name": "GPT-3.5 (davinci-002)\n", "code": null}, {"id": 368108, "name": "Segatron-XL large, M=384 + HCP", "code": null}, {"id": 371963, "name": "Make-A-Scene", "code": null}, {"id": 273166, "name": "Chinchilla", "code": null}, {"id": 273167, "name": "PaLM (540B)", "code": null}, {"id": 306186, "name": "DALL\u00b7E 2", "code": null}, {"id": 368730, "name": "BERT-RBP", "code": null}, {"id": 343968, "name": "Stable Diffusion (LDM-KL-8-G)", "code": null}, {"id": 268378, "name": "Sparse all-MLP", "code": null}, {"id": 306187, "name": "Flamingo", "code": null}, {"id": 306188, "name": "OPT-175B", "code": null}, {"id": 306190, "name": "UL2", "code": null}, {"id": 306191, "name": "Gato", "code": null}, {"id": 306192, "name": "Imagen", "code": null}, {"id": 371857, "name": "GPT-2 Medium (FlashAttention)", "code": null}, {"id": 368110, "name": "Tranception", "code": null}, {"id": 368096, "name": "DITTO", "code": null}, {"id": 368373, "name": "CoCa", "code": null}, {"id": 306194, "name": "Parti", "code": null}, {"id": 369037, "name": "ProGen2-xlarge", "code": null}, {"id": 306195, "name": "Minerva (540B)", "code": null}, {"id": 368133, "name": "CodeT5-large", "code": null}, {"id": 306196, "name": "NLLB", "code": null}, {"id": 368746, "name": "BLOOM-176B", "code": null}, {"id": 369022, "name": "ESM2-15B", "code": null}, {"id": 369033, "name": "OmegaPLM", "code": null}, {"id": 354863, "name": "AlexaTM 20B", "code": null}, {"id": 365992, "name": "GLM-130B", "code": null}, {"id": 368716, "name": "BlenderBot 3", "code": null}, {"id": 369347, "name": "BEIT-3", "code": null}, {"id": 368374, "name": "PaLI", "code": null}, {"id": 349174, "name": "Whisper", "code": null}, {"id": 369185, "name": "DiffDock", "code": null}, {"id": 371829, "name": "AlphaTensor", "code": null}, {"id": 369025, "name": "GenSLM", "code": null}, {"id": 369171, "name": "Flan-PaLM 540B", "code": null}, {"id": 368739, "name": "U-PaLM (540B)", "code": null}, {"id": 369151, "name": "eDiff-I", "code": null}, {"id": 368006, "name": "Mogrifier RLSTM (WT2)", "code": null}, {"id": 369198, "name": "InternImage", "code": null}, {"id": 369545, "name": "EVA-01", "code": null}, {"id": 368109, "name": "Galactica", "code": null}, {"id": 368735, "name": "Fusion in Encoder", "code": null}, {"id": 368082, "name": "AR-LDM", "code": null}, {"id": 368749, "name": "Discriminator Guidance", "code": null}, {"id": 371482, "name": "Vega v2", "code": null}, {"id": 368726, "name": "CaLM", "code": null}, {"id": 368002, "name": "Hybrid H3-2.7B", "code": null}, {"id": 369034, "name": "VALL-E", "code": null}, {"id": 371511, "name": "DreamerV3", "code": null}, {"id": 369009, "name": "Nucleotide Transformer", "code": null}, {"id": 368732, "name": "Ankh_large", "code": null}, {"id": 368354, "name": "DDPM-IP (CelebA)", "code": null}, {"id": 369196, "name": "BLIP-2 (Q-Former)", "code": null}, {"id": 369193, "name": "ViT-22B", "code": null}, {"id": 367499, "name": "LLaMA-65B", "code": null}, {"id": 368750, "name": "DiT-XL/2", "code": null}, {"id": 369016, "name": "AudioGen", "code": null}, {"id": 367636, "name": "Falcon-40B", "code": null}, {"id": 372333, "name": "GPT-4 (Jun 2023)", "code": null}, {"id": 372308, "name": "GPT-4 (Mar 2023)", "code": null}, {"id": 368069, "name": "PanGu-\u03a3", "code": null}, {"id": 371534, "name": "SigLIP 400M", "code": null}, {"id": 369142, "name": "VideoMAE V2", "code": null}, {"id": 367081, "name": "BloombergGPT", "code": null}, {"id": 369040, "name": "Segment Anything Model", "code": null}, {"id": 368359, "name": "Incoder-6.7B", "code": null}, {"id": 369173, "name": "DINOv2", "code": null}, {"id": 369186, "name": "LLaVA", "code": null}, {"id": 368130, "name": "StarCoder", "code": null}, {"id": 365387, "name": "PaLM 2", "code": null}, {"id": 369334, "name": "InstructBLIP", "code": null}, {"id": 366449, "name": "ONE-PEACE", "code": null}, {"id": 368325, "name": "PaLI-X", "code": null}, {"id": 369030, "name": "HyenaDNA", "code": null}, {"id": 369204, "name": "Pangu-Weather", "code": null}, {"id": 367509, "name": "InternLM", "code": null}, {"id": 369010, "name": "xTrimoPGLM -100B", "code": null}, {"id": 367637, "name": "Claude 2", "code": null}, {"id": 368743, "name": "Llama 2-70B", "code": null}, {"id": 368747, "name": "Llama 2-7B", "code": null}, {"id": 369002, "name": "AudioLM", "code": null}, {"id": 369035, "name": "GGNN", "code": null}, {"id": 369042, "name": "PeptideBERT", "code": null}, {"id": 367533, "name": "Jais", "code": null}, {"id": 367528, "name": "Swift", "code": null}, {"id": 369508, "name": "Falcon-180B", "code": null}, {"id": 370131, "name": "Amazon Titan", "code": null}, {"id": 368729, "name": "FinGPT-13B", "code": null}, {"id": 371966, "name": "RoseTTAFold All-Atom (RFAA)", "code": null}, {"id": 368129, "name": "CODEFUSION (Python)", "code": null}, {"id": 369998, "name": "ChatGLM3-6B", "code": null}, {"id": 368349, "name": "Skywork-13B", "code": null}, {"id": 368740, "name": "Yi-34B", "code": null}, {"id": 368376, "name": "Grok-1", "code": null}, {"id": 369331, "name": "LLaVA 1.5", "code": null}, {"id": 370174, "name": "CogVLM-17B", "code": null}, {"id": 369014, "name": "MultiBand Diffusion", "code": null}, {"id": 372317, "name": "RoFormer", "code": null}, {"id": 369144, "name": "SPHINX (Llama 2 13B)", "code": null}, {"id": 369349, "name": "Volcano 13B", "code": null}, {"id": 369547, "name": "GraphCast", "code": null}, {"id": 369019, "name": "Nemotron-3-8B", "code": null}, {"id": 368751, "name": "Inflection-2", "code": null}, {"id": 368736, "name": "Qwen-72B", "code": null}, {"id": 369506, "name": "Gemini 1.0 Ultra", "code": null}, {"id": 369174, "name": "Llama Guard", "code": null}, {"id": 369027, "name": "Mixtral 8x7B", "code": null}, {"id": 371814, "name": "VILA-13B", "code": null}, {"id": 369518, "name": "CogAgent", "code": null}, {"id": 369532, "name": "FunSearch", "code": null}, {"id": 371483, "name": "nekomata-14b", "code": null}, {"id": 372331, "name": "GQA-8-XXL", "code": null}, {"id": 371236, "name": "Qwen1.5-72B", "code": null}, {"id": 371971, "name": "Stable Diffusion 3", "code": null}, {"id": 369522, "name": "MegaScale (Production)", "code": null}, {"id": 369880, "name": "Mistral Large", "code": null}, {"id": 371244, "name": "Aramco Metabrain AI", "code": null}, {"id": 369561, "name": "Inflection-2.5", "code": null}, {"id": 369524, "name": "MM1-30B", "code": null}, {"id": 369883, "name": "DBRX", "code": null}, {"id": 370170, "name": "Reka Core", "code": null}, {"id": 369516, "name": "Llama 3-70B", "code": null}, {"id": 371961, "name": "GenCast", "code": null}, {"id": 371528, "name": "VILA1.5-13B", "code": null}, {"id": 371962, "name": "AlphaFold 3", "code": null}, {"id": 370141, "name": "Yi-Large", "code": null}, {"id": 371524, "name": "Octo-Base", "code": null}, {"id": 371545, "name": "ALLaM\u00a0adapted 70B", "code": null}, {"id": 370106, "name": "Qwen2-72B", "code": null}, {"id": 371839, "name": "Llama-3.1-Nemotron-70B-Instruct", "code": null}, {"id": 370246, "name": "OpenVLA", "code": null}, {"id": 369982, "name": "Nemotron-4 340B", "code": null}, {"id": 370239, "name": "DeepSeek-Coder-V2 236B", "code": null}, {"id": 370244, "name": "Claude 3.5 Sonnet", "code": null}, {"id": 369981, "name": "ESM3 (98B)", "code": null}, {"id": 370155, "name": "Llama 3.1-405B", "code": null}, {"id": 370085, "name": "Mistral Large 2", "code": null}, {"id": 370093, "name": "AFM-on-device", "code": null}, {"id": 370105, "name": "AFM-server", "code": null}, {"id": 371536, "name": "LLaVA-OV-72B", "code": null}, {"id": 371231, "name": "Grok-2", "code": null}, {"id": 370561, "name": "DeepSeek-V2.5", "code": null}, {"id": 371474, "name": "Qwen2.5-32B", "code": null}, {"id": 371475, "name": "Qwen2.5 Instruct (72B)", "code": null}, {"id": 370534, "name": "Qwen2.5-72B", "code": null}, {"id": 371840, "name": "Telechat2-115B", "code": null}, {"id": 371540, "name": "Llama 3.2 11B", "code": null}, {"id": 371329, "name": "Movie Gen Video", "code": null}, {"id": 371653, "name": "RDT-1B", "code": null}, {"id": 370972, "name": "CHAI-1", "code": null}, {"id": 371481, "name": "Yi-Lightning", "code": null}, {"id": 371240, "name": "NVLM-D 72B", "code": null}, {"id": 371234, "name": "NVLM-H 72B", "code": null}, {"id": 371232, "name": "NVLM-X 72B", "code": null}, {"id": 371248, "name": "Doubao-pro", "code": null}, {"id": 371242, "name": "Hunyuan-Large", "code": null}, {"id": 371249, "name": "Amazon Nova Pro", "code": null}, {"id": 371535, "name": "Llama 3.3 70B", "code": null}, {"id": 371364, "name": "EXAONE 3.5 32B", "code": null}, {"id": 371328, "name": "DeepSeek-V3", "code": null}, {"id": 371370, "name": "DeepSeek-R1", "code": null}, {"id": 371654, "name": "Eagle 2", "code": null}, {"id": 371865, "name": "Grok 3", "code": null}, {"id": 371367, "name": "Claude 3.7 Sonnet", "code": null}, {"id": 371369, "name": "GPT-4.5", "code": null}, {"id": 371365, "name": "QwQ-32B", "code": null}, {"id": 371513, "name": "Hunyuan-TurboS", "code": null}, {"id": 371472, "name": "EXAONE Deep 32B", "code": null}, {"id": 372634, "name": "DeepSeek-V3 (Mar 2025)", "code": null}, {"id": 371514, "name": "Llama 4 Behemoth (preview)", "code": null}, {"id": 371860, "name": "Llama 4 Maverick", "code": null}, {"id": 371843, "name": "Llama 4 Scout", "code": null}, {"id": 371529, "name": "Pangu Ultra", "code": null}, {"id": 371991, "name": "Qwen3-235B-A22B", "code": null}, {"id": 371939, "name": "Seed1.5-VL", "code": null}, {"id": 372346, "name": "DeepSeek-R1 (May 2025)", "code": null}, {"id": 371850, "name": "FGN", "code": null}, {"id": 371820, "name": "Grok 4", "code": null}, {"id": 371815, "name": "Kimi K2", "code": null}, {"id": 371817, "name": "EXAONE 4.0 (32B)", "code": null}, {"id": 371987, "name": "Qwen3-Coder-480B-A35B", "code": null}, {"id": 372632, "name": "Qwen3-235B-A22B (Jul 2025)", "code": null}, {"id": 372358, "name": "Qwen3-235B-A22B-Thinking (Jul 2025)", "code": null}, {"id": 372366, "name": "GLM-4.5", "code": null}, {"id": 371847, "name": "gpt-oss-120b", "code": null}, {"id": 371855, "name": "gpt-oss-20b", "code": null}, {"id": 371945, "name": "GPT-5", "code": null}, {"id": 372167, "name": "LongCat-Flash", "code": null}, {"id": 371992, "name": "Qwen3-Max", "code": null}, {"id": 372330, "name": "AgentFounder-30B", "code": null}, {"id": 372169, "name": "Qwen3-Omni-30B-A3B", "code": null}, {"id": 372365, "name": "GLM-4.6", "code": null}, {"id": 372318, "name": "Ling-1T", "code": null}, {"id": 372341, "name": "Kimi K2 Thinking", "code": null}, {"id": 372433, "name": "Olmo 3", "code": null}, {"id": 372369, "name": "GLM-4.7", "code": null}, {"id": 372363, "name": "K-EXAONE", "code": null}]}}, "origins": [{"id": 14136, "title": "Parameter, Compute and Data Trends in Machine Learning", "descriptionSnapshot": "We update this chart with the latest available data from our source every month.\n\nThe authors selected the AI systems for inclusion based on the following necessary criteria:\n\u2014 Have an explicit learning component\n\u2014 Showcase experimental results\n\u2014 Advance the state of the art\n\nIn addition, the systems had to meet at least one of the following notability criteria:\n\u2014 Paper has more than 1000 citations\n\u2014 Historical importance\n\u2014 Important state-of-the-art advance\n\u2014 Deployed in a notable context\n\nThe authors note that: \"For new models (from 2020 onward) it is harder to assess these criteria, so we fall back to a subjective selection. We refer to models meeting our selection criteria as 'milestone models.\"\n", "producer": "Epoch AI", "citationFull": "Epoch AI, \u2018Parameter, Compute and Data Trends in Machine Learning\u2019. Published online at epochai.org. Retrieved from: \u2018https://epoch.ai/data/epochdb/visualization\u2019 [online resource]", "urlMain": "https://epoch.ai/mlinputs/visualization", "urlDownload": "https://epoch.ai/data/epochdb/notable_ai_models.csv", "dateAccessed": "2026-03-07", "datePublished": "2025", "license": {"url": "https://creativecommons.org/licenses/by/4.0/", "name": "CC BY 4.0"}}]}