How to log data in a .csv file

With STM32CubeMonitor tool, logged data are saved in a .stcm file. This format allows to :

• Post-process the data in a different flow
• Import directly the data in a ST chart

Nevertheless, some manipulations are allowed by a .csv file and not by a .stcm file, such as :

• When a data issue occurs, the user can easily analyse the data with a .csv file
• Share the data with other users

1. Convert .stcm log file to a .csv file

It is possible to convert the generated .stcm log file in a .csv to easily manage your data. In order to perform this, import the following flow into the STM32CubeMonitor workspace :

 [{"id":"7e9c50ed.2ab67","type":"subflow","name":"Group variables in a .csv file","info":"","category":"STMicroelectronics","in":[{"x":60,"y":80,"wires":[{"id":"c7f149ae.894818"}]}],"out":[{"x":840,"y":360,"wires":[{"id":"802c75f2.9f7898","port":0}]}],"env":[{"name":"nbVar","type":"num","value":"","ui":{"icon":"font-awesome/fa-cogs","label":{"en-US":"Number of Variables"},"type":"input","opts":{"types":["num"]}}},{"name":"delete_time","type":"bool","value":"false","ui":{"icon":"font-awesome/fa-clock-o","label":{"en-US":"Single Time"},"type":"input","opts":{"types":["bool"]}}}],"color":"#3CB4E6","icon":"node-red/join.svg"},{"id":"7b54efc3.0009e","type":"function","z":"7e9c50ed.2ab67","name":"Associate values with same timestamp","func":"let i = 0;\nlet j = 0;\nlet n = 0;\nlet msg1; \nlet msgArray = new Array();\nlet nbVar = msg.payload.length;\n\nfor(i = 0; i < nbVar; i++){\n    \n    let underArrayLength = msg.payload[i].length;\n    \n    for(j = 0; j < underArrayLength; j++){\n            \n            for(n = 0; n < nbVar; n++){\n                msg1 = msg.payload[n][j];\n                msgArray.push(msg1);\n            }\n    }\n    msg.payload = msgArray;\n    return msg;\n}","outputs":1,"noerr":0,"x":650,"y":80,"wires":[["b3b24fdd.b01b9"]]},{"id":"b3b24fdd.b01b9","type":"split","z":"7e9c50ed.2ab67","name":"","splt":"\\n","spltType":"str","arraySplt":"${nbVar}","arraySpltType":"len","stream":false,"addname":"","x":870,"y":80,"wires":[["cf754130.7de2a"]]},{"id":"ca91d405.238ea8","type":"function","z":"7e9c50ed.2ab67","name":"Format data","func":"let output=\"\";\n\nif(typeof msg.payload == 'number') {\n    output = msg.payload;\n    \n}else if(typeof msg.payload == 'string'){\n    output = msg.payload;\n}else{\n    output = msg.payload.x + \";\"+ msg.payload.y;\n}\n\nmsg.payload = output ;\n\nreturn msg;","outputs":1,"noerr":0,"x":790,"y":200,"wires":[["802c75f2.9f7898"]]},{"id":"802c75f2.9f7898","type":"join","z":"7e9c50ed.2ab67","name":"Join consecutive messages from the number of monitored variables","mode":"custom","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":";","joinerType":"str","accumulate":false,"timeout":"","count":"${nbVariables}","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":380,"y":360,"wires":[[]]},{"id":"c7f149ae.894818","type":"join","z":"7e9c50ed.2ab67","name":"Join [nbVar] consecutives messages","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":";","joinerType":"str","accumulate":false,"timeout":"","count":"${nbVariables}","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":290,"y":80,"wires":[["7b54efc3.0009e"]]},{"id":"cf754130.7de2a","type":"function","z":"7e9c50ed.2ab67","name":"Delete duplicate timestamps if user wants","func":"let i;\nlet delDup = env.get(\"delete_time\");\nlet nbVar = env.get(\"nbVariables\");\nlet msgArray = new Array();\n\nif(delDup === false){\n    return msg;\n}else{\n    \n    msgArray.push(msg.payload[0]);\n    \n    for(i=1; i < nbVar; i++){\n        \n        msgArray.push(msg.payload[i].y);\n        \n    }\n    \n    msg.payload = msgArray;\n    return msg;\n    \n}\n\n\n","outputs":1,"noerr":0,"x":260,"y":200,"wires":[["91613227.a8eb8"]]},{"id":"91613227.a8eb8","type":"split","z":"7e9c50ed.2ab67","name":"","splt":"\\n","spltType":"str","arraySplt":"1","arraySpltType":"len","stream":false,"addname":"","x":570,"y":200,"wires":[["ca91d405.238ea8"]]},{"id":"c468592c.a96b08","type":"subflow:7e9c50ed.2ab67","z":"94e67f1.803b48","name":"Group variables in a .csv file","env":[{"name":"nbVar","value":"2","type":"num"},{"name":"nbVariables","value":"2","type":"num"}],"x":980,"y":360,"wires":[["9019184f.2360e8"]]}]

Menu > Import > Clipboard and Copy/Paste the .json formatted flow in the area, click on Import. The flow is now imported into the STM32CubeMonitor workspace.

1.1. How does the flow works ?

The conversion will be started on the press of the Start button. The read file node parse each line of the log file and send it to the "Group variables" subflow. Then the subflow format the data in a ".csv way" and send it to the file node. This file node create a new file : the .csv file corresponding to its .stcm.

1.2. Configure the flow

Before doing the first file conversion, the flow need to be configured :

• "Set file name" : in the "Set msg.filename to ..." set to [STCM_File_Absolute_Path]. Here is an example of configuration :

• "Format CSV Header" : this node is used to define the .csv file column names. Fill the "Set msg.payload to ..." field by putting the .csv column names in the "to" field. The user needs to choose to display one timestamp for all variables monitored at the same time, or to display each timestamp for each variable. The data are acquired like this :

Time_Data1, Value_Data1, Time_Data2, Value_Data2 ...

Example of configuration :

• To display one timestamp for all variables monitored at the same time :

• To display the timestamp of each variable :

/!\ The column names have to be set in the same order than the acquired data one (order giving by the variable node of the acquisition flow) /!\

The subflow configuration will be described in the following part.

1.3. "Group variables" subflow

1.3.1. Parameters

To configure this subflow two parameters are needed :

• nbVar : the number of monitored variables
• Delete_Time : a boolean value to define if the user wants to display (or not) the timestamp for each variable (default value sets to False) :
  • True : Only one timestamp
  • False : One timestamp for each variable

1.3.2. How does it work ?

This subflow associates the values with the same timestamp and then let the choice to the user to display (or not) the duplicates timestamp. Finally, it will format the data in a ".csv way".

2. Log your data directly in a .csv file

This method allows the user to directly generate a .csv file without passing through a .stcm file. To perform this, import the following flow in the STM32CubeMonitor workspace :

[{"id":"7e9c50ed.2ab67","type":"subflow","name":"Group variables in a .csv file","info":"","category":"STMicroelectronics","in":[{"x":60,"y":80,"wires":[{"id":"c7f149ae.894818"}]}],"out":[{"x":840,"y":360,"wires":[{"id":"802c75f2.9f7898","port":0}]}],"env":[{"name":"nbVar","type":"num","value":"","ui":{"icon":"font-awesome/fa-cogs","label":{"en-US":"Number of Variables"},"type":"input","opts":{"types":["num"]}}},{"name":"delete_time","type":"bool","value":"false","ui":{"icon":"font-awesome/fa-clock-o","label":{"en-US":"Single Time"},"type":"input","opts":{"types":["bool"]}}}],"color":"#3CB4E6","icon":"node-red/join.svg"},{"id":"7b54efc3.0009e","type":"function","z":"7e9c50ed.2ab67","name":"Associate values with same timestamp","func":"let i = 0;\nlet j = 0;\nlet n = 0;\nlet msg1; \nlet msgArray = new Array();\nlet nbVar = msg.payload.length;\n\nfor(i = 0; i < nbVar; i++){\n    \n    let underArrayLength = msg.payload[i].length;\n    \n    for(j = 0; j < underArrayLength; j++){\n            \n            for(n = 0; n < nbVar; n++){\n                msg1 = msg.payload[n][j];\n                msgArray.push(msg1);\n            }\n    }\n    msg.payload = msgArray;\n    return msg;\n}","outputs":1,"noerr":0,"x":650,"y":80,"wires":[["b3b24fdd.b01b9"]]},{"id":"b3b24fdd.b01b9","type":"split","z":"7e9c50ed.2ab67","name":"","splt":"\\n","spltType":"str","arraySplt":"${nbVar}","arraySpltType":"len","stream":false,"addname":"","x":870,"y":80,"wires":[["cf754130.7de2a"]]},{"id":"ca91d405.238ea8","type":"function","z":"7e9c50ed.2ab67","name":"Format data","func":"let output=\"\";\n\nif(typeof msg.payload == 'number') {\n    output = msg.payload;\n    \n}else if(typeof msg.payload == 'string'){\n    output = msg.payload;\n}else{\n    output = msg.payload.x + \";\"+ msg.payload.y;\n}\n\nmsg.payload = output ;\n\nreturn msg;","outputs":1,"noerr":0,"x":790,"y":200,"wires":[["802c75f2.9f7898"]]},{"id":"802c75f2.9f7898","type":"join","z":"7e9c50ed.2ab67","name":"Join consecutive messages from the number of monitored variables","mode":"custom","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":";","joinerType":"str","accumulate":false,"timeout":"","count":"${nbVariables}","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":380,"y":360,"wires":[[]]},{"id":"c7f149ae.894818","type":"join","z":"7e9c50ed.2ab67","name":"Join [nbVar] consecutives messages","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":";","joinerType":"str","accumulate":false,"timeout":"","count":"${nbVariables}","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":290,"y":80,"wires":[["7b54efc3.0009e"]]},{"id":"cf754130.7de2a","type":"function","z":"7e9c50ed.2ab67","name":"Delete duplicate timestamps if user wants","func":"let i;\nlet delDup = env.get(\"delete_time\");\nlet nbVar = env.get(\"nbVariables\");\nlet msgArray = new Array();\n\nif(delDup === false){\n    return msg;\n}else{\n    \n    msgArray.push(msg.payload[0]);\n    \n    for(i=1; i < nbVar; i++){\n        \n        msgArray.push(msg.payload[i].y);\n        \n    }\n    \n    msg.payload = msgArray;\n    return msg;\n    \n}\n\n\n","outputs":1,"noerr":0,"x":260,"y":200,"wires":[["91613227.a8eb8"]]},{"id":"91613227.a8eb8","type":"split","z":"7e9c50ed.2ab67","name":"","splt":"\\n","spltType":"str","arraySplt":"1","arraySpltType":"len","stream":false,"addname":"","x":570,"y":200,"wires":[["ca91d405.238ea8"]]},{"id":"91d68244.047e7","type":"subflow","name":"Select .csv variable","info":"","category":"STMicroelectronics","in":[{"x":220,"y":120,"wires":[{"id":"e9aa15c3.2c15f8"}]}],"out":[{"x":1180,"y":120,"wires":[{"id":"d287b70e.45b4c8","port":0}]}],"env":[{"name":"varfilter","type":"str","value":"","ui":{"icon":"font-awesome/fa-cog","label":{"en-US":"Variable name"},"type":"input","opts":{"types":["str"]}}}],"color":"#3CB4E6","icon":"node-red/arrow-in.svg"},{"id":"d287b70e.45b4c8","type":"change","z":"91d68244.047e7","name":"keep only table of duplets","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.variabledata","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":950,"y":120,"wires":[[]]},{"id":"e9aa15c3.2c15f8","type":"switch","z":"91d68244.047e7","name":"filter variable","property":"payload.variablename","propertyType":"msg","rules":[{"t":"eq","v":"varfilter","vt":"env"}],"checkall":"true","repair":false,"outputs":1,"x":390,"y":120,"wires":[["c88ffe31.b60d9"]]},{"id":"c88ffe31.b60d9","type":"function","z":"91d68244.047e7","name":"define the .csv header ","func":"if(msg.payload.variabledata[0].y === undefined){\n    msg.payload.variabledata[0].x = 'timestamp' + \"_\" +  msg.payload.variablename;\n    msg.payload.variabledata[0].y = msg.payload.variablename;\n    return msg;\n}else{\n    return msg;\n}\n","outputs":1,"noerr":0,"x":660,"y":120,"wires":[["d287b70e.45b4c8"]]},{"id":"b724a6a7.177628","type":"tab","label":"Create .csv log file","disabled":false,"info":""},{"id":"f474eca8.7b351","type":"ui_chartst","z":"b724a6a7.177628","group":"1b8f9f84.b59fb","name":"myChart","order":4,"width":15,"height":9,"chartType":"line","curveType":"linear","duration":"10","ymin":"","ymax":"","x":620,"y":300,"wires":[]},{"id":"af306ec4.f0fad","type":"acquisition out","z":"b724a6a7.177628","name":"myProbe_Out","probeconfig":"3ab5d867.e37ba8","x":680,"y":140,"wires":[]},{"id":"be311bcb.2d3738","type":"acquisition in","z":"b724a6a7.177628","name":"myProbe_In","probeconfig":"3ab5d867.e37ba8","x":230,"y":380,"wires":[["d42b7c5c.e9b9"],[]]},{"id":"672c629e.55c89c","type":"ui_button","z":"b724a6a7.177628","name":"","group":"1b8f9f84.b59fb","order":3,"width":5,"height":1,"passthru":false,"label":"Clear Graphs","tooltip":"","color":"","bgcolor":"","icon":"","payload":"","payloadType":"str","topic":"clear","x":410,"y":300,"wires":[["f474eca8.7b351"]]},{"id":"fddfae1a.63681","type":"ui_button","z":"b724a6a7.177628","name":"","group":"1b8f9f84.b59fb","order":1,"width":5,"height":1,"passthru":false,"label":"START Acquisition","tooltip":"","color":"","bgcolor":"","icon":"","payload":"","payloadType":"str","topic":"start","x":250,"y":100,"wires":[["9767ad34.f46fd"]]},{"id":"7aa5d3b.ac0f92c","type":"ui_button","z":"b724a6a7.177628","name":"","group":"1b8f9f84.b59fb","order":2,"width":5,"height":1,"passthru":true,"label":"STOP Acquisition","tooltip":"","color":"","bgcolor":"","icon":"","payload":"","payloadType":"str","topic":"stop","x":250,"y":180,"wires":[["9767ad34.f46fd"]]},{"id":"d42b7c5c.e9b9","type":"processing","z":"b724a6a7.177628","groupname":"MyVariables","groupid":"9767ad34.f46fd","expressions":[],"statistics":[],"logmode":"no","x":490,"y":400,"wires":[["f474eca8.7b351","fdcd4a27.d79ec8","c6e46d72.82fc5"],[]]},{"id":"b8f3a432.74d7e8","type":"file","z":"b724a6a7.177628","name":"","filename":"c:\\Temp\\Log_File.csv","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":1300,"y":420,"wires":[[]]},{"id":"3c143b95.1aabb4","type":"comment","z":"b724a6a7.177628","name":"link the subflows in the .exe variables order","info":"","x":720,"y":480,"wires":[]},{"id":"c83e71e8.a4d32","type":"comment","z":"b724a6a7.177628","name":"Enter the number of variables connected","info":"","x":1020,"y":380,"wires":[]},{"id":"fdcd4a27.d79ec8","type":"subflow:91d68244.047e7","z":"b724a6a7.177628","name":"Data1","env":[{"name":"varfilter","value":"data1","type":"str"}],"x":690,"y":380,"wires":[["3f392198.19ae2e"]]},{"id":"c6e46d72.82fc5","type":"subflow:91d68244.047e7","z":"b724a6a7.177628","name":"Data2","env":[{"name":"varfilter","value":"data2","type":"str"}],"x":690,"y":440,"wires":[["3f392198.19ae2e"]]},{"id":"3f392198.19ae2e","type":"subflow:7e9c50ed.2ab67","z":"b724a6a7.177628","name":"Group variables in a .csv file","env":[{"name":"nbVar","value":"2","type":"num"},{"name":"nbVariables","value":"4","type":"num"}],"x":1000,"y":420,"wires":[["b8f3a432.74d7e8"]]},{"id":"93517461.eeea38","type":"comment","z":"b724a6a7.177628","name":"Enter absolute path of the file","info":"","x":1320,"y":460,"wires":[]},{"id":"9767ad34.f46fd","type":"variables","z":"b724a6a7.177628","groupname":"MyVariables","accesspoint":0,"execonfig":"","variablelist":[],"triggerstartmode":"manual","triggername":"","triggerthreshold":"","frequency":"","frequencyType":"0","snapshotheader":"","mode":"direct","lastImportedTime":-1,"openStatus":false,"x":490,"y":140,"wires":[["af306ec4.f0fad"]]},{"id":"1b8f9f84.b59fb","type":"ui_group","z":"","name":"Chart","tab":"e4a22138.efd5b","order":1,"disp":true,"width":15,"collapse":false},{"id":"3ab5d867.e37ba8","type":"probe","z":"","probeid":"066FFF485457725187092531","probename":"ST-Link v2-1B (...92531)","protocol":"SWD","frequency":"1.8 MHz - Default","probeproperty":"{\"SWD\":[\"4.6 MHz\",\"1.8 MHz - Default\",\"950 kHz\",\"400 kHz\",\"150 kHz\"],\"JTAG\":[\"18 MHz\",\"9 MHz\",\"4.5 MHz\",\"2.25 MHz\",\"1.12 MHz - Default\",\"560 kHz\",\"280 kHz\",\"140 kHz\"]}","probeversion":"ST Link firmware version V2.J36","connectionType":"p2p"},{"id":"e4a22138.efd5b","type":"ui_tab","z":"","name":"Home","icon":"dashboard","disabled":false,"hidden":false}]

Menu > Import > Clipboard and Copy/Paste the .json formatted flow in the area, click on Import. The flow is now imported into the STM32CubeMonitor workspace.

2.1. How does the flow work ?

The data sent at the output of the processing node are then sent to the "select .csv variables" subflow (one subflow per monitored variable). This subflow filters each variable and send it to another subflow : "Group variables". This last subflow will group, sort and format the data to create the .csv file.

2.2. Configure your flow

Before doing the first file conversion, the flow need to be configured :

• For each monitored variable, add a “select .csv variables” subflow connected to the processing node output

/!\ Be careful, you need to link your subflow inputs in the same order than your processing node output /!\

• In the “file node”, set the absolute path of the .csv file you want to create.

The subflows configurations will be described in the following parts.

2.3. “Select .csv variables” subflow

2.3.1. Parameters

To configure this subflow, only one parameter is needed : the name of the variable to filter. One subflow is needed for each variable you want to log. In the “varfilter” field put the exact name of the filtered variable.

/!\ Be careful, the variable name has to be exactly the same as that in the processing node (without space before and after). You can copy/paste it from the processing node /!\

2.3.2. How does it work ?

This subflow has several roles :

• Filters each variables by its name
• Format the header of the .csv file
• Finally, sends all the data to the “Group variables” subflow.

2.4. “Group variables” subflow

2.4.1. Parameters

To configure this subflow two parameters are needed :

• nbVar : the number of monitored variables
• Delete_Time : a boolean value to define if the user wants to display (or not) the timestamp for each variable (default value sets to False) :
  • True : Only one timestamp
  • False : One timestamp for each variable

2.4.2. How does it work ?

This subflow associates the values with the same timestamp and then let the choice to the user to display (or not) the duplicates timestamp. Finally, it will format the data in a ".csv way".

2.5. Open your .csv file

To find the .csv file, go in the folder defined in the “file node”. In this folder, a new .csv file has been created.