Energy insight app? CSV to Graph

There are all kind of apps out there that can connect to your solar panels and show you nice graphs on your energy position: they share the habit to collect your data (often in PRC) and use it for other purposes as well. Same applies for your energy meters, which you can set up to report more often (near real-time), but where do your data go to?

It would be nice if we could somehow collect our own data and show it on our own cloud instead.

There is a great variety on hardware and software out there, so it would be a huge effort to build a one-system-fits-all solution. IMHO it can be split in two phases:

  • gathering the data into usable files
  • use thes files to give insight

For a start I started polling my solar inverter with a cronjob on my cloudserver/webuser:

# a new day starts
1 0 * * * /usr/bin/echo "Date|Time|Power[W]|Yield/day[*10Wh]|Cumulative[/10 kWh]|Alert|Age[min]" >> /data/user/files/Solar/solar$( date +'\%Y\%m\%d' ).csv
# poll inverter to get data every 10 minutes
*/10 * * * * /usr/bin/bash -c /var/lib/wwwrun/csv.solar.day
# compact logfile from the day before
58 23 * * * /usr/bin/gzip -9 /data/user/files/Solar/solar$( date --date="2 day ago" +\%Y\%m\%d ).csv
# a new month started
45 23 1 * * /usr/bin/echo "Date|Yield/day[*10Wh]|Cumulative[/10 kWh]" >> /data/user/files/Solar/solar$( date +'\%Y\%m' ).csv
# poll inverter once more to get result of today and add result to month file
50 23 * * * /usr/bin/bash -c /var/lib/wwwrun/csv.solar.month

Of course the “magic” in my specific use case is done in the csv.solar.* scripts, but from there we could have a NextCloud generic situation.

These two files are written during the day:
/data/user/files/Solar/solarYYYYMMDD.csv with content

Date|Time|Power[W]|Yield/day[*10Wh]|Cumulative[/10 kWh]|Alert|Age[min]
20221220|10:20:04|25|1|66832||1
20221220|10:30:04|37|1|66832||1
20221220|10:40:05|37|1|66832||11
20221220|10:50:04|32|2|66832||1
20221220|11:00:04|41|3|66832||1
20221220|11:10:04|28|4|66832||1
20221220|11:20:04|18|4|66832||6
20221220|11:30:04|22|4|66832||1

So: From the moment the solar panels start to produce per 10 minutes a line is added with the current data. This can be used to produce a graph per day

/data/user/files/Solar/solarYYYYMM.csv with content

Date|Yield/day[*10Wh]|Cumulative[/10 kWh]
20221219|36|66832

So: Per day a line is added with the production for that day, so you can also graph the totals per month.

Those two files are in the data directory of a user on the cloud but how can I create graphs on my cloudserver from these rather generic csv files?

  1. How can these files become automatically “known” to the user (currently they are there but not visible in the cloud)
  2. How can I draw a graph based on those csv files to show?

Or has this already been made somehow?

It seems to me that you are looking for this app:

https://apps.nextcloud.com/apps/analytics

As you can see in the documentation ( Home · Rello/analytics Wiki · GitHub ), it should allow to obtain the graphs and to update them as needed

Great hint, however, where crontab is writing the files somewhere (to be more precise into a user area files folder), how can I see them in nextcloud?

Do I have to write the files onto my webserver instead? But then they are visible from anywhere…

This is a way to go:


Thanks for the hint.
Next challenge is to have Nextcloud poll the inverter using the http web crawler.