DATA CENTRIC AI vs MODEL CENTRIC AI: WHICH APPROACH TO GO AFTER IN MACHINE LEARNING

ABSTRACT:

An important topic in the study and practice of artificial intelligence (AI) is the schism between data-centric and model-centric AI. These strategies have different benefits and drawbacks, which has led to a great deal of discussion among scholars and professionals. This article presents a critical assessment of the two paradigms and argues that the data-centric approach is better. Although the value of model-centric AI is duly recognised, this article suggests that a data-centric approach is more important for building strong AI systems. This claim is supported in light of the computing requirements, performance consequences, and data's function in model building. A thorough grasp of the subject is aided by the citation of scholarly works that support this viewpoint.

INTRODUCTION:

Artificial intelligence (AI) has altered a wide range of industries, from healthcare to banking, by leveraging data to generate actionable insights and facilitate decision-making. In the field of AI, two opposing techniques have emerged: data-centric AI and model-centric AI. Data-centric AI prioritises data quality, volume, and preparation approaches, whereas model-centric AI concentrates on the creation of complicated algorithms and structures for extracting patterns from data. This article will examine various techniques and argue that the data-centric approach is superior in modern AI research and implementation.

DATA CENTRIC AI:

Data-centric AI emphasises the need for high-quality, diversified, and large datasets in driving AI performance. This method prioritises data preparation approaches such as data augmentation, normalisation, and feature engineering to improve AI system efficacy. Data-centric AI aims to reduce biases, improve generalisation, and increase model resilience by methodically selecting and improving datasets.

Furthermore, data-centric AI places a high value on continuous monitoring and adaptation to changing data environments. It recognises that the relevance and freshness of the underlying data have a significant impact on the quality of AI models. Thus, in addition to initial dataset curation, continuing data validation and enrichment activities are essential components of the data-centric strategy. Continuous review ensures that the AI system stays current with data trends and patterns, increasing its flexibility and reliability over time. This iterative refinement decreases the risk of model decay and makes it easier to identify and mitigate emergent biases or anomalies, ensuring that AI systems remain durable and successful in dynamic situations.

MODEL CENTRIC AI:

Model-centric AI, on the other hand, focuses on creating sophisticated algorithms and architectures to extract nuanced patterns from data. This strategy focuses on improving model performance by refining neural network topologies, optimisation algorithms, and regularisation techniques. Model-centric AI advocates argue that advances in model complexity and expressiveness are critical to pushing the bounds of AI capabilities.

Furthermore, model-centric AI advocates propose that the development of complex algorithms allows for the extraction of deeper and more nuanced insights from data, resulting in improved forecast accuracy and decision-making capabilities. Model-centric approaches try to find complicated links and patterns that traditional statistical methods may not be able to discern. Moreover, advances in model complexity enable the integration of numerous data modalities, such as text, images, and audio, allowing AI systems to tackle multidimensional problems more effectively. Thus, proponents of model-centric AI argue that continual innovation in algorithmic design is required to push the limits of AI capabilities and solve more complex real-world problems.

COMPUTATIONAL DEMANDS:

The computing demands of either approach are an important aspect of the debate over data-centric vs model-centric AI. Model-centric techniques frequently require significant computational resources to train complex models, undertake hyperparameter optimisation, and evaluate model performance. In contrast, data-centric approaches are less computationally intensive, concentrating on effective data pretreatment and augmentation techniques. This makes data-centric AI more accessible to researchers and practitioners with low computational power.

Furthermore, the computational efficiency of data-centric AI enables scalability and deployment in resource-constrained situations. Data-centric techniques make it possible to deploy AI solutions across a variety of platforms, including edge devices and IoT systems, by reducing the computing burden during the training and inference phases. This scalability not only helps to democratise AI technologies but also makes them more useful in real-world circumstances where computational resources are constrained or spread. Furthermore, the decreased computational overhead of data-centric techniques leads to lower operational costs and shorter time-to-market for AI applications, making them more appealing for industry adoption. As a result, the computational efficiency of data-centric AI not only meets practical resource limits but also promotes creativity and accessibility in AI deployment.

PERFORMANCE IMPLICATIONS:

The performance implications of data-centric and model-centric AI are critical elements in determining the effectiveness of AI systems. Proponents of model-centric AI frequently emphasise the importance of increasing model complexity, arguing that elaborate algorithms can find deeper insights and patterns within data, resulting in higher performance on benchmark datasets and real-world applications. These proponents contend that the intricacy of model topologies, combined with thorough tweaking of hyperparameters and regularisation approaches, considerably improves prediction accuracy and generalisation capabilities.

However, empirical research has presented persuasive data that contradicts the concept that model complexity alone assures higher performance. Research has demonstrated that rigorously curated datasets and appropriate preprocessing approaches can have a considerable impact on AI performance, sometimes to the point where results are comparable among models. This emphasises how important data quality and preprocessing are in deciding the efficacy of AI systems. Data-centric techniques create a solid foundation for model building by focusing on acquiring high-quality, diverse, and representative datasets first. Furthermore, using rigorous preprocessing approaches like data augmentation, normalisation, and feature engineering can reduce biases, improve generalisation, and strengthen model robustness.

ROLE OF DATA IN MODEL DEVELOPMENT:

At the heart of data-centric AI is a comprehensive understanding of data as the foundation of model building. High-quality datasets are essential for developing robust models that can display improved generalisation and robustness to domain transitions. Data-centric approaches promote model development by ensuring the integrity, diversity, and relevance of the data utilised for model training.

Moreover, data-centric AI employs an iterative refinement process in which datasets are continuously updated, enriched, and validated to improve their representativeness and adaptability to changing situations (Krizhevsky et al., 2012). This iterative method not only improves the efficiency and durability of existing models but also provides the framework for future advances in AI. By focusing on the acquisition, curation, and refinement of high-quality datasets, academics and practitioners not only improve current performance but also create a fertile field for innovation and the investigation of fresh AI paradigms.

CONCLUSION:

To summarise, the debate between data-centric AI and model-centric AI emphasises the complex nature of AI research and implementation. While both approaches have significant advantages, this article claims that the data-centric approach is critical in contemporary AI endeavours. By emphasising data quality, preprocessing procedures, and iterative refinement, data-centric AI promotes the creation of robust and generalizable AI systems. However, it is critical to recognise the complementary nature of model-centric approaches and to use breakthroughs in both paradigms to move the science of AI forward.

REFERENCES:

1. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete Problems in AI Safety. arXiv preprint arXiv:1606.06565.

2. Bengio, Y., Courville, A., & Vincent, P. (2015). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8), 1798-1828.

3. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning (Vol. 1). MIT press Cambridge.

4. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

5. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).

6. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436-444.

7. Lipton, Z. C. (2018). The mythos of model interpretability. arXiv preprint arXiv:1606.03490.

8. Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019). Do ImageNet classifiers generalize to ImageNet?. arXiv preprint arXiv:1902.10811 (underlined as this is the deal-breaking research on why the data-centric approach is better).

9. Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., ... & Young, M. (2015). Hidden technical debt in machine learning systems. In Advances in neural information processing systems (pp. 2503-2511).

10. Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015). Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3156-3164).

11. Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2017). Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530.

12. Radečić, D. (2022, March 23). Data-centric vs. Model-centric AI? The Answer is Clear. Medium. https://towardsdatascience.com/data-centric-vs-model-centric-ai-the-answer-is-clear-4b607c58af67

13. O. H. Hamid, "From Model-Centric to Data-Centric AI: A Paradigm Shift or Rather a Complementary Approach?," 2022 8th International Conference on Information Technology Trends (ITT), Dubai, United Arab Emirates, 2022, pp. 196-199.

Preprint

Preprint

DATA CENTRIC AI vs MODEL CENTRIC AI: WHICH APPROACH TO GO AFTER IN MACHINE LEARNING