Missing datapoints in exported CSV's #125
Johannes (2019-07-11 11:45)
· 766 view(s)
· 4 comment(s)
Hi,
Thanks for a great program.
I am recording (demo) data from a Philips Intellivue MP50. Seemingly works fine, but when I Export to CSV, with epoch equal to the highest sample rate, the data is not equally spaced.
There does not seem to be any pattern. Later in the file, it looks like this:
vitaldb2017 (2019-07-12 01:57)
This is related to the difference in output resolution between the two wave data.
In the attached figure, the data point interval is 0.002 seconds and 30 data points are 0.06 seconds.
For ABP, there are 8 values in 30 data points (0.06 seconds) and 19 values in ECG.
The output resolution of the two waves is roughly calculated as follows:
ABP: 8/0.06 =133 Hz
ECG: 19/0.06 = 316 Hz
Therefore, if you want to represent both data at regular intervals in one sheet, you will need to output the least common multiple of the time interval frequency of the two variables.
Of course, the empty cells that occur need imputation separately.
In the attached figure, the data point interval is 0.002 seconds and 30 data points are 0.06 seconds.
For ABP, there are 8 values in 30 data points (0.06 seconds) and 19 values in ECG.
The output resolution of the two waves is roughly calculated as follows:
ABP: 8/0.06 =133 Hz
ECG: 19/0.06 = 316 Hz
Therefore, if you want to represent both data at regular intervals in one sheet, you will need to output the least common multiple of the time interval frequency of the two variables.
Of course, the empty cells that occur need imputation separately.
Johannes (2019-07-12 08:20)
My original post was actually a bit longer. There is probably some limit to the length.
The ECG is recorded at 500 Hz. Later in the same file, it looks like this:
https://imgur.com/a/tIan1MI
The ECG is recorded at 500 Hz. Later in the same file, it looks like this:
https://imgur.com/a/tIan1MI
vitaldb2017 (2019-07-16 06:37)
Philips monitors and several other types of patient monitors use high-speed serial communication to send high-resolution waveform data over serial communications. Your USB-Serial converter may not be able to handle the communication speed. In this case, there may be a delay in data transmission through serial communication and data loss may occur.
Anyway, we are going to do some more research on the Philips monitor communication so that we can fix the bug.
Anyway, we are going to do some more research on the Philips monitor communication so that we can fix the bug.
Johannes (2019-08-07 06:58)
I have now written a python script to extract individual waveforms from the binary file (i could not make the script on Github work). These waveforms do not have missing data, so the problem is not in the serial communication.