Can LLMs Assist Computer Education? an Empirical Case Study of DeepSeek
Dongfu Xiao ⋆ \star, Chen Gao ⋆ \star, Zhengquan Luo, Chi Liu, Sheng Shen · 2025
Read Can LLMs Assist Computer Education? an Empirical Case Study of DeepSeek by Dongfu Xiao ⋆ \star, Chen Gao ⋆ \star, Zhengquan Luo, Chi Liu, Sheng Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-00421.txt.
Introduction
Dongfu Xiao and four co-authors test whether a large language model can help teach computer networking. Their 2025 case study puts DeepSeek-V3 to work on certification-style simulation questions and on real security questions from Chinese network engineers. They probe the model's steadiness along three lines: whether a role prompt changes its answers, whether it copes across languages, and whether it repeats itself.
The model proved steady on all three checks, with stable accuracy in both the source and translated sets. A clear split appeared by task type. It handled factual recall well but strained on tasks that demand layered reasoning. Gaps also showed with image-based input and the hardest topics. The authors see real value for network security teaching, with clear limits to respect.
Catalog record
| CREATOR | Dongfu Xiao ⋆ \star, Chen Gao ⋆ \star, Zhengquan Luo, Chi Liu, Sheng Shen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-00421
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can LLMs Assist Computer Education? an Empirical Case Study of DeepSeek Dongfu Xiao ⋆ \star, Chen Gao ⋆ \star, Zhengquan Luo, Chi Liu, Sheng Shen Abstract This study presents an empirical case study to assess the efficacy and reliability of DeepSeek-V3, an emerging large language model, within the context of computer education. The evaluation employs both CCNA simulation questions and real-world inquiries concerning computer network security posed by Chinese network engineers. To ensure a thorough evaluation, diverse dimensions are considered, encompassing role dependency, cross-linguistic proficiency, and answer reproducibility, accompanied by statistical analysis. The findings demonstrate that the model performs consistently, regardless of whether prompts include a role definition or not. In addition, its adaptability across languages is confirmed by maintaining stable accuracy in both original and translated datasets. A distinct contrast emerges between its performance on lower-order factual recall tasks and higher-order reasoning exercises, which underscores its strengths in retrieving information and its limitations in complex analytical tasks. Although DeepSeek-V3 offers considerable practical value for network security education, challenges remain in its capability to process multimodal data and address highly intricate topics. These results provide valuable insights for future refinement of large language models in specialized professional environments. 1 Introduction Recent advancements in large language models (LLMs) have opened substantial opportunities to transform professional education in computer networking and security. These models exhibit advanced capabilities in processing and articulating intricate technical concepts [ 20 , 21 ] , making them potent educational tools that can complement conventional learning methods [ 18 ] . Their capacity to produce precise, context-aware explanations of networking and security principles [ 11 ] , coupled with the generation of tailored instructional content [ 14 ] , establishes LLMs as valuable resources for developing professional expertise. Although initial studies have shown LLMs’ proficiency in applying domain-specific knowledge, their efficacy in comprehensive networking education—particularly concerning complex technical concepts and practical problem-solving scenarios—necessitates systematic evaluation using established professional benchmarks [ 20 ] . Notably, next-generation LLMs like DeepSeek-V3 present significant untapped potential in this domain. Built on the Transformer architecture, DeepSeek-V3 has achieved groundbreaking progress in natural language processing through large-scale data training, demonstrating unique advantages in code generation and mathematical reasoning. Its 128K context window and dynamic knowledge retrieval capabilities are particularly promising for handling complex networking tasks with long-sequence dependencies, including protocol analysis and topology configuration [ 13 ] . This combination of technical features suggests new possibilities for advancing both research and education in computer networking. To rigorously evaluate the practical effectiveness of DeepSeek-V3 in computer networking, a systematic assessment across diverse network scenarios is necessary. For this study, we selected two authoritative test datasets: the latest simulated question bank from the Cisco Certified Network Associate (CCNA) exam—a widely recognized industry standard—and a subset of real questions from China’s 2022-2023 Network Engineer certification (part of the Computer Technology and Software Professional Qualification Examination, commonly known as the Soft Exam) [ 9 , 5 ] . The CCNA exam covers fundamental networking concepts, including IP addressing, routing, and switching, while the Network Engineer exam assesses broader competencies such as network planning, security, and management. Both examinations integrate theoretical networking principles with practical problem-solving, presenting challenging questions that test both conceptual and applied knowledge. Given their comprehensive coverage, technical depth, and varying difficulty levels, these exams serve as robust benchmarks for evaluating DeepSeek-V3’s capabilities in computer networking. By analyzing the model’s performance on these standardized assessments, we can assess its proficiency in understanding, applying, and solving network-related problems. The findings will provide valuable insights for further optimization and real-world deployment of this language model in networking applications. 2 Related Works 2.1 LLM LLMs constitute a category of deep neural networks characterized by their massive scale (typically comprising billions to trillions of parameters), which demonstrate exceptional proficiency in language comprehension and generation, thereby enabling seamless human-machine interaction through natural language interfaces [ 4 ] . The pervasive adoption of LLMs in recent years has driven significant progress across multiple disciplines, including healthcare [ 19 , 2 ] , educational technology [ 3 ] , computer science research [ 12 ] , and cybersecurity [ 8 ] . However, these performance improvements have been accompanied by exponentially increasing computational costs during training. Current state-of-the-art models exhibit substantial variations in training expenditures: Google’s Gemini Ultra ranks as the most expensive at $191 million, followed by OpenAI’s GPT-4 at $78 million. Notably, DeepSeek achieves competitive performance with significantly lower training costs of only $5.6 million [ 17 ] . Among existing LLMs, DeepSeek stands out for its exceptional cost-efficiency while remaining open-source, contributing to its widespread adoption across various application domains. 2.2 Evaluation of LLMs in Professional Domains The application of LLMs to standardized and professional certification exams has grown significantly across multiple disciplines. This systematic assessment examines their competence in domain-specific knowledge evaluation, reasoning precision, and practical utility [ 2 , 16 , 15 ] . Current research in this area reveals a strong emphasis on medical applications, particularly in evaluations such as Medical Licensing Examinations [ 2 ] and Nuclear Medicine Physician Board Examinations [ 19 ] . In computer science, [ 16 ] analyzed ChatGPT-4’s performance on Brazil’s National Undergraduate Computer Science Examination. The predominant evaluation approach involves curating authoritative exam questions from standardized tests and benchmarking LLMs’ ability to answer them. Assessments commonly employ single-choice and multiple-choice formats to rigorously measure domain-specific problem-solving capabilities, quantified by accuracy in addressing discipline-related queries [ 16 ] . Additionally, multiple studies have explored LLM performance across different languages [ 2 , 19 , 15 ] . Nevertheless, no existing work has conducted a systematic evaluation of DeepSeek’s performance on network engineer certification examinations. 3 Methodology 3.1 DeepSeek DeepSeek’s language model achieves state-of-the-art performance in natural language processing, with empirical results demonstrating robust capabilities in both text generation and complex reasoning tasks. Benchmark evaluations confirm DeepSeek-V3’s advanced language and reasoning capabilities, with officially reported accuracies of 92.3% on GSM8K mathematical reasoning (5-shot), 79.6% on English AGIEval (0-shot), and 90.1% on Chinese C-Eval (5-shot) [ 1 ] . These results, derived from standardized evaluations under controlled protocols, position the model among the top-performing open-weight systems and validate its reliability for this study’s methodology. This study employs the official DeepSeek API for experimental analysis [ 7 ] . The DeepSeek-V3 model utilizes a mixture of experts (MoE) architecture comprising 671 billion total parameters with approximately 37 billion activated per token [ 13 ] . Pretrained on 14.8 trillion tokens of curated multilingual data from web texts, technical documents, and academic publications, it demonstrates robust performance across diverse domains while maintaining efficient inference [ 13 ] . 3.2 Question Selection and Classification To systematically assess large language models’ capabilities in computer network security, we selected two authoritative test banks: the official question repository from China’s Software Professional and Technical Qualification Examination (200 questions total) and the CCNA simulation test bank (380 questions total). Since DeepSeek-V3 primarily processes textual data, we excluded image-based questions, resulting in a final evaluation set comprising 122 questions from the Chinese examination and 199 from the CCNA test bank. These curated question sets strictly align with the respective examination syllabi, ensuring comprehensive coverage of fundamental computer network security concepts. The