A multimodal dataset for environmental occupancy detection Authors: Guilherme Dall’Agnol Deconto, Avelino Francisco Zorzo, Roben Castagna Lunardi, Rafael C. Cardoso, Leonardo dos Santos Teixeira Published: 2026 ABSTRACT This article presents a multimodal dataset collected in a real smart environment located at the Pontifical Catholic University of Rio Grande do Sul (PUCRS), Brazil. The dataset documents environmental, electrical, and device-interaction measurements collected from a heterogeneous Internet of Things (IoT) deployment composed of commercial smart devices and a custom ESP32-based sensing node. Environmental variables include carbon dioxide concentration, temperature, relative humidity, light intensity, and sound pressure level. Electrical measurements include instantaneous power, voltage, current, and device state indicators collected from smart sockets, switches, and a dedicated server. Additional metadata include device status indicators.The dataset also includes a human-verified ground_truth variable indicating the number of occupants in the room at each timestamp. This variable was produced through a two-stage annotation procedure in which YOLO26m detections were used as initial occupancy count suggestions and all corresponding camera images were subsequently reviewed manually to confirm or correct the final labels.Data were collected continuously under routine academic workspace operation, reflecting natural occupancy fluctuations, restricted access control, and heterogeneous network conditions. The custom sensing node was programmed in Arduino C++ and exposed measurements via HTTP requests, while commercial devices were polled through the local Tuya LAN protocol using the TuyAPI library. All records were stored in structured JSON format using a common timestamp assigned at the end of each acquisition cycle. Each record represents the aggregation of values polled within the same 10 s collection window, rather than perfectly simultaneous measurements acquired at the exact same instant.The dataset is publicly available in a repository and includes raw measurements, device logs, and documentation describing hardware configuration, and acquisition procedures. These data can support research in occupancy detection, energy usage analysis, anomaly detection, and smart-building experimentation. The dataset also provides a documented example of a real-world IoT deployment suitable for replication or comparative studies. Specifications TableSubjectComputer SciencesSpecific subject areaMultimodal IoT sensing for smart indoor occupancy monitoring.Type of data.json, .csvData collectionData were collected from commercial IoT devices and a custom ESP32 sensing node installed in a smart office environment. Measurements were retrieved through local polling of Tuya-based devices using the TuyAPI library and through HTTP requests to the ESP32 sensing node and were aggregated by a local server every 10 seconds. Environmental data, electrical consumption, and device state indicators were stored as JSON records in a MongoDB database and exported in JSON Lines format. Occupancy ground truth was obtained through manual review of camera images, with YOLO26m detections used only as initial pre-annotation suggestions.Data source locationPontifical Catholic University of Rio Grande do Sul, Av. Ipiranga, 6681, Partenon, Porto Alegre/RS, Brazil, CEP (ZIP Code) 90619-900.Data accessibilityRepository name: ZenodoData identification number: 10.5281/zenodo.19184829Direct URL to data: https://doi.org/10.5281/zenodo.19184829Related research articleNone. SubjectComputer SciencesSpecific subject areaMultimodal IoT sensing for smart indoor occupancy monitoring.Type of data.json, .csvData collectionData were collected from commercial IoT devices and a custom ESP32 sensing node installed in a smart office environment. Measurements were retrieved through local polling of Tuya-based devices using the TuyAPI library and through HTTP requests to the ESP32 sensing node and were aggregated by a local server every 10 seconds. Environmental data, electrical consumption, and device state indicators were stored as JSON records in a MongoDB database and exported in JSON Lines format. Occupancy ground truth was obtained through manual review of camera images, with YOLO26m detections used only as initial pre-annotation suggestions.Data source locationPontifical Catholic University of Rio Grande do Sul, Av. Ipiranga, 6681, Partenon, Porto Alegre/RS, Brazil, CEP (ZIP Code) 90619-900.Data accessibilityRepository name: ZenodoData identification number: 10.5281/zenodo.19184829Direct URL to data: https://doi.org/10.5281/zenodo.19184829Related research articleNone. Value of the Data •This dataset [1] provides a real-world multimodal smart-office record collected from heterogeneous commercial IoT devices and a custom ESP32 sensing node operating under everyday conditions. Unlike smaller benchmark datasets focused mainly on environmental variables, this dataset combines environmental sensing, device-state logs, and occupancy-related signals within the same deployment.•The data are useful for evaluating occupancy estimation methods in realistic settings affected by network instability, missing values, vendor-device heterogeneity, and routine human activity. This makes the dataset suitable not only for occupancy detection, but also for anomaly detection, smart-environment monitoring, and robustness studies in practical IoT scenarios.•Unlike occupancy datasets focused mainly on a limited number of ambient sensors and more controlled acquisition settings, the present dataset captures a broader range of smart-environment interactions, including connected devices, actuation states, and operational irregularities observed in a live academic workspace. •This dataset [1] provides a real-world multimodal smart-office record collected from heterogeneous commercial IoT devices and a custom ESP32 sensing node operating under everyday conditions. Unlike smaller benchmark datasets focused mainly on environmental variables, this dataset combines environmental sensing, device-state logs, and occupancy-related signals within the same deployment. This dataset [1] provides a real-world multimodal smart-office record collected from heterogeneous commercial IoT devices and a custom ESP32 sensing node operating under everyday conditions. Unlike smaller benchmark datasets focused mainly on environmental variables, this dataset combines environmental sensing, device-state logs, and occupancy-related signals within the same deployment. •The data are useful for evaluating occupancy estimation methods in realistic settings affected by network instability, missing values, vendor-device heterogeneity, and routine human activity. This makes the dataset suitable not only for occupancy detection, but also for anomaly detection, smart-environment monitoring, and robustness studies in practical IoT scenarios. The data are useful for evaluating occupancy estimation methods in realistic settings affected by network instability, missing values, vendor-device heterogeneity, and routine human activity. This makes the dataset suitable not only for occupancy detection, but also for anomaly detection, smart-environment monitoring, and robustness studies in practical IoT scenarios. •Unlike occupancy datasets focused mainly on a limited number of ambient sensors and more controlled acquisition settings, the present dataset captures a broader range of smart-environment interactions, including connected devices, actuation states, and operational irregularities observed in a live academic workspace. Unlike occupancy datasets focused mainly on a limited number of ambient sensors and more controlled acquisition settings, the present dataset captures a broader range of smart-environment interactions, including connected devices, actuation states, and operational irregularities observed in a live academic workspace. Background The dataset was compiled as part of an effort to document and analyze a real smart-office environment used by the CONSEG research group at PUCRS. The initial motivation was to create a controlled yet functioning space where IoT devices and environmental sensors could be deployed and observed under routine activity. This setting enabled the systematic collection of data reflecting device interactions, environmental conditions, and human presence events within a restricted-access workplace. From a methodological standpoint, the dataset emerged from the need to support research on sensor-based occupancy characterization and digital forensics scenarios. Establishing a reproducible and well-documented data acquisition process was a key objective, allowing researchers to examine how heterogeneous IoT components operate in this kind of environment. If associated with an original research article, this dataset adds value by providing the underlying raw measurements, device logs, and acquisition procedures required to replicate experiments, validate analytical methods, or extend the scope of the published study. It also enables independent exploration of the smart-office infrastructure beyond the specific analyses presented in the research article. Data Description The repository contains two files: final_dataset_csv.csv and final_dataset_json.json. These files store all measurements collected from the smart-office deployment during the data acquisition period. The dataset is stored in both JSON Lines format, where each line corresponds to an independent JSON object representing a single acquisition cycle and a CSV file with the same content. Records were generated at 10-second intervals during the acquisition period, producing approximately 360 records per hour. Each line corresponds to a timestamped acquisition record. Device-level measurements were flattened into a tabular schema in which each key represents a specific sensor measurement or device state. Device prefixes (e.g., server_, pfsense_, socket_, lamp_, switch_) identify the device from which each measurement originates. The dataset includes a ground_truth field representing the human-verified number of occupants in the room for each timestamped record. Occupancy labels were produced through a two-stage annotation procedure based on camera images collected during the acquisition period. First, YOLO26m-based person detection was used to generate initial occupancy count suggestions. Then, all corresponding camera images associated with the dataset records were manually inspected by human annotators, who confirmed or corrected the number of visible occupants. Therefore, the ground_truth values represent manually verified occupancy counts rather than raw object detection outputs. The dataset includes a mixture of numerical and binary variables. Numerical variables represent continuous environmental and electrical measurements such as temperature, power consumption, voltage, current, and sound levels. Binary variables represent device states such as lamp and switch states. The dataset contains approximately 230,976 timestamped records and 22 variables, occupying approximately 128,3 MB in JSON Lines format and 34,8 MB in CSV format.MetricUnitDescriptioncreatedAtISO 8601 timestampTimestamp of the acquisition recordground_truthcountHuman-verified number of occupants in the roomco2ppmCarbon dioxide concentrationhumidity%Relative humiditytemperature_1°CTemperature measurement from sensor channel 1temperature_2°CTemperature measurement from sensor channel 2lux_1luxLight intensity from sensor channel 1lux_2luxLight intensity from sensor channel 2sounddBSound pressure level measured in the roomserver_cur_currentmAInstantaneous current consumption of the serverserver_cur_powerWInstantaneous power consumption of the serverserver_cur_voltageVInstantaneous voltage measurement of the serverlamp_switch_ledbinary (0/1)Lamp power statepfsense_cur_currentmAInstantaneous current consumption of the pfSense devicepfsense_cur_powerWInstantaneous power consumption of the pfSense devicepfsense_cur_voltageVInstantaneous voltage measurement of the pfSense deviceswitch_channel_1binary (0/1)Switch state, channel 1switch_channel_2binary (0/1)Switch state, channel 2switch_channel_3binary (0/1)Switch state, channel 3socket_cur_currentmAInstantaneous current consumption of the socketsocket_cur_powerWInstantaneous power consumption of the socketsocket_cur_voltageVInstantaneous voltage measurement of the socket MetricUnitDescriptioncreatedAtISO 8601 timestampTimestamp of the acquisition recordground_truthcountHuman-verified number of occupants in the roomco2ppmCarbon dioxide concentrationhumidity%Relative humiditytemperature_1°CTemperature measurement from sensor channel 1temperature_2°CTemperature measurement from sensor channel 2lux_1luxLight intensity from sensor channel 1lux_2luxLight intensity from sensor channel 2sounddBSound pressure level measured in the roomserver_cur_currentmAInstantaneous current consumption of the serverserver_cur_powerWInstantaneous power consumption of the serverserver_cur_voltageVInstantaneous voltage measurement of the serverlamp_switch_ledbinary (0/1)Lamp power statepfsense_cur_currentmAInstantaneous current consumption of the pfSense devicepfsense_cur_powerWInstantaneous power consumption of the pfSense devicepfsense_cur_voltageVInstantaneous voltage measurement of the pfSense deviceswitch_channel_1binary (0/1)Switch state, channel 1switch_channel_2binary (0/1)Switch state, channel 2switch_channel_3binary (0/1)Switch state, channel 3socket_cur_currentmAInstantaneous current consumption of the socketsocket_cur_powerWInstantaneous power consumption of the socketsocket_cur_voltageVInstantaneous voltage measurement of the socket Missing values may occur due to device disconnections, temporary network issues, or unsuccessful polling attempts. In the JSON version of the dataset, missing measurements are represented by absent fields. In the CSV version, missing measurements appear as blank cells. Missing values were not replaced by zeros or by the last known value. Experimental Design, Materials and Methods Data were acquired in the CONSEG Lab Smart Office, located on the sixth floor of Building 32 at PUCRS, Porto Alegre, Brazil. The room is a glass-walled office used by undergraduate and graduate students, and professors for meetings and daily work activities, with restricted access. The environment includes standard office furniture (desks, chairs, cabinets), multiple power sockets, wired network points, and existing fluorescent tube lighting connected to smart wall switches. Temperature is centrally controlled by the building’s HVAC system, with a dedicated room sensor but no local thermostat control. The smart office deployment combines commercial IoT devices and a custom sensing unit. The commercial devices include an Echo Dot (4th generation) smart speaker, a smart wall switch (EWS 1003) for the main lights, three smart sockets (EWS 301), an Asus RT-AX82U router/switch, a Tapo C200 camera, an EPGG20 smart lamp, a ZigBee 3.0 hub, and a ZigBee motion sensor (SNZB-03). Each device was configured using the manufacturer’s smartphone application to register it in the corresponding cloud ecosystem and associate it with the laboratory Wi-Fi network. Devices were paired with the smart speaker to enable voice-based control and to centralize automations. All IoT devices were connected to a dedicated network segment provided by an Asus RT-AX82U router, which is itself connected to the custom lab PFSense server which is connected to the university network. This separated network simplifies device management and avoids interference with the main institutional Wi-Fi used by laptops and personal devices. All commercial IoT devices communicate with their manufacturers’ cloud services via the Internet; local commands issued through the smart speaker or apps are relayed through those cloud services before reaching the devices. Initial deployment required adjustments to university network rules to allow outbound connections to the relevant cloud endpoints and device registration services. In addition to commercial devices, a custom sensing node was assembled using an ESP32 microcontroller and attached environmental sensors to measure light and CO₂ concentration. This node is physically installed inside the room and continuously collects measurements at a fixed sampling interval defined in the firmware. The code is written in Arduino C++ and timestamps all readings using the ESP32 internal clock, synchronized periodically with an NTP server or a local time reference. Measurements are exposed via an HTTP endpoint. When requested, the ESP32 collects sensor readings and returns them in JSON format. In addition to the ESP32 microcontroller, the custom sensing node incorporated an SCD30 CO2 sensor module. According to the manufacturer, this sensor is factory-calibrated and linearized. However, no independent pre-deployment calibration against a certified reference CO2 instrument was performed in this study. Consequently, the CO2 values should be interpreted as raw measurements from a commercial IoT sensing deployment rather than reference-grade calibrated environmental measurements. A dedicated laboratory server (Intel i5-8500 processor, 16 GB RAM, Debian 12) was used to aggregate measurements from commercial IoT devices and the custom ESP32 sensing node. Data were collected at fixed intervals through local polling of Tuya-based devices using the TuyAPI library and through HTTP requests to the ESP32 sensing node and were stored in JSON format in a MongoDB database. Timestamps are stored in ISO 8601 format with timezone offset and can be normalized to UTC during processing. A Node.js and Python-based collection service periodically queried devices and appended records to the database. Data were collected using periodic polling. The collection service executed an acquisition cycle every 10 seconds, querying all available devices and storing the returned values as a single timestamped record. Data were collected between December 4, 2023, and January 16, 2024, in a continuously operating academic smart-office environment. The devices used in this experiment were mostly Tuya-based. Although these devices were initially configured through the manufacturer’s cloud ecosystem, periodic data acquisition was performed locally through the Tuya LAN protocol using the TuyAPI library. Thus, the recorded measurements reflect the device state returned through local polling at the time of request. Room occupancy labels were produced through a two-stage annotation procedure based on camera images collected during the acquisition period. First, person detection was performed using the YOLO26m object detection model, pretrained on the COCO dataset, to generate initial occupancy count suggestions for each image. These model-generated suggestions were used only as a pre-annotation aid. After this initial step, all camera images associated with the dataset were manually inspected by human annotators. During this review, the number of people visible in the room at each timestamp was checked and corrected when necessary. The final value stored in the dataset is the ground_truth variable, which represents the human-verified number of occupants in the room, rather than the raw output of the object detection model. The YOLO26m model was used only to support the annotation workflow and was not used as the final source of occupancy labels. No additional fine-tuning was performed. The final ground_truth values were assigned after manual verification of the corresponding camera images. Limitations Several limitations are associated with the data described in this article. Data collection occurred in a naturally functioning academic workspace, leading to irregular occupancy patterns influenced by meetings, class schedules, and holidays. As a result, some periods contain sparse activity, while others present dense event sequences. Certain commercial IoT devices rely on cloud services, and intermittent connectivity or vendor-side outages occasionally produced gaps in the event logs. A few devices required manual intervention for configuration or reactivation, producing short intervals without measurements. The smart power strip originally planned for the deployment was defective during the collection period and was therefore excluded from the final dataset. The custom ESP32-based sensor node operated independently from the commercial ecosystem. Some sensors provide only binary or low-resolution measurements, which can limit granularity. The dataset preserves the raw values returned by the deployed sensors and devices. No post-hoc correction, smoothing, or replacement of unexpected sensor readings was applied. Therefore, sensor faults, transient measurement errors, or device-specific inaccuracies may be reflected in the dataset. Users should consider task-specific preprocessing, filtering, or validation procedures when using these measurements for model development or benchmarking. Bias may also arise from the restricted-access nature of the laboratory, where the population is composed primarily of researchers and students rather than a general office workforce. Ethics Statement All individuals who participated in the research environment and could be involved in the data collection procedures were informed about the study objectives, procedures, and data handling practices, and provided their informed consent prior to participation. CRediT Author Statement Guilherme Dall’Agnol Deconto: Methodology, Software, Writing – Original Draft; Avelino Francisco Zorzo: Conceptualization, Supervision; Roben Castagna Lunardi: Validation, Writing - Review & Editing, Project administration; Rafael C. Cardoso: Writing - Review & Editing, Resources; Leonardo dos Santos Teixeira: Data Curation.