Data Analytics v1.0.0

Rello · December 1, 2019, 8:57pm

I am happy to announce the first release fo Data Analytics

https://apps.nextcloud.com/apps/analytics

The roadmap is filled. I am happy for any feedback or suggestion.

Happy first advent

kesselb · December 2, 2019, 10:59am

Thank you!

Probably your profile needs an update now

Rello · December 2, 2019, 11:25am

I know
@jospoortvliet needs to help here

zx81 · December 9, 2019, 8:16pm

Nice app!

Do you have plans about integrate more that one graph per page? I mean, sometimes is useful to have a week view, month view, year view, all in the same page.

Something strange, If i generate a CSV with two columns the graph doesn’t works (only show the X axis), I allways need to add a first column (no matters the data) and after that the graph works (using columns 2 and 3), is this a bug?

Rello · December 9, 2019, 8:33pm

Hello,
I am planning for a more sophisticated reporting frontend for DA - but its on a longer horizon. e.g. printable layouts. But I need to finish the basis first. like your next point

not really a bug. missing feature.
currently DA is missing full flexibility on the source data. it works perfect with data like Account/Date/Value.
when using less dimensions, the first one needs to be “blanked” as the highchart depends on a number in the dimension 3.

More flexibility is on its way! Medium roadmap would also include more dimensions or star-scheme like attributes. lets see

…I am just finishing customisable threshold, which trigger NC notifications because I need them for one of my use cases…

jakobssystems · December 10, 2019, 5:27pm

Thank you very much for this app. Maybe I am too stupid but I am failing in my attempts to send data via curl. I am always recieving error message

url":"/apps/analytics/api/1.0/adddata/3","message":{"Exception":"ParseError","Message":"syntax error, unexpected ')'"

when trying to send:

curl -u login:pass -d '{"dimension1": "x", "dimension2": "x", "dimension3": "333,3"}' -X POST -H "Content-Type: application/json" https://cloud.mydomain.de/apps/analytics/api/1.0/adddata/myID

(replacing login,pass and myID with matching values and mydomain.de with my domain)

I am missing something?

Rello · December 10, 2019, 7:14pm

did you set pass to an app-token and not your normal NC user password?
you need to generate a token under settings => security

jakobssystems · December 10, 2019, 8:01pm

Hi Rello, thank you for your rapid response.
Of course I’ve created an App Token. Do you need any logs? Or do you need a Teamviewer Session?

Rello · December 10, 2019, 9:18pm

Hello,
i just tried with my instance and nothing obvious broke there.
what does netcloud.log say?

jakobssystems · December 11, 2019, 6:05pm

here as screenshot and raw text with asterix* for private data:

{“reqId”:“",“level”:3,“time”:“2019-12-10T17:01:26+00:00”,“remoteAddr”:"”,“user”:"****",“app”:“index”,“method”:“POST”,“url”:"/apps/analytics/api/1.0/adddata/3",“message”:{“Exception”:“ParseError”,“Message”:“syntax error, unexpected ‘)’”,“Code”:0,“Trace”:[{“file”:"/var/www/cloud.jakobssystems.de/lib/composer/composer/ClassLoader.php",“line”:322,“function”:“Composer\Autoload\includeFile”,“args”:["/var/www/cloud.jakobssystems.de/apps/analytics/lib//Controller/ApiDataController.php"]},{“function”:“loadClass”,“class”:“Composer\Autoload\ClassLoader”,“type”:"->",“args”:[“OCA\Analytics\Controller\ApiDataController”]},{“function”:“spl_autoload_call”,“args”:[“OCA\Analytics\Controller\ApiDataController”]},{“file”:"/var/www/cloud.jakobssystems.de/lib/private/AppFramework/Utility/SimpleContainer.php",“line”:97,“function”:"__construct",“class”:“ReflectionClass”,“type”:"->",“args”:[“OCA\Analytics\Controller\ApiDataController”]},{“file”:"/var/www/cloud.jakobssystems.de/lib/private/AppFramework/Utility/SimpleContainer.php",“line”:114,“function”:“resolve”,“class”:“OC\AppFramework\Utility\SimpleContainer”,“type”:"->",“args”:[“OCA\Analytics\Controller\ApiDataController”]},{“file”:"/var/www/cloud.jakobssystems.de/lib/private/AppFramework/DependencyInjection/DIContainer.php",“line”:414,“function”:“query”,“class”:“OC\AppFramework\Utility\SimpleContainer”,“type”:"->",“args”:[“OCA\Analytics\Controller\ApiDataController”]},{“file”:"/var/www/cloud.jakobssystems.de/lib/private/AppFramework/DependencyInjection/DIContainer.php",“line”:385,“function”:“queryNoFallback”,“class”:“OC\AppFramework\DependencyInjection\DIContainer”,“type”:"->",“args”:[“OCA\Analytics\Controller\ApiDataController”]},{“file”:"/var/www/cloud.jakobssystems.de/lib/private/AppFramework/App.php",“line”:113,“function”:“query”,“class”:“OC\AppFramework\DependencyInjection\DIContainer”,“type”:"->",“args”:[“OCA\Analytics\Controller\ApiDataController”]},{“file”:"/var/www/cloud.jakobssystems.de/lib/private/AppFramework/Routing/RouteActionHandler.php",“line”:47,“function”:“main”,“class”:“OC\AppFramework\App”,“type”:"::",“args”:[“OCA\Analytics\Controller\ApiDataController”,“addData”,{“class”:“OC\AppFramework\DependencyInjection\DIContainer”},{“datasetId”:“3”,"_route":“analytics.ApiData.addData”}]},{“function”:"__invoke",“class”:“OC\AppFramework\Routing\RouteActionHandler”,“type”:"->",“args”:[{“datasetId”:“3”,"_route":“analytics.ApiData.addData”}]},{“file”:"/var/www/cloud.jakobssystems.de/lib/private/Route/Router.php",“line”:297,“function”:“call_user_func”,“args”:[{“class”:“OC\AppFramework\Routing\RouteActionHandler”},{“datasetId”:“3”,"_route":“analytics.ApiData.addData”}]},{“file”:"/var/www/cloud.jakobssystems.de/lib/base.php",“line”:1000,“function”:“match”,“class”:“OC\Route\Router”,“type”:"->",“args”:["/apps/analytics/api/1.0/adddata/3"]},{“file”:"/var/www/cloud.jakobssystems.de/index.php",“line”:42,“function”:“handleRequest”,“class”:“OC”,“type”:"::",“args”:[]}],“File”:"/var/www/cloud.jakobssystems.de/apps/analytics/lib/Controller/ApiDataController.php",“Line”:50,“CustomMessage”:"–"},“userAgent”:“curl/7.55.1”,“version”:“17.0.1.1”,“id”:“5df12e8c11d0c”}

The only diffrence as far as I can see is that in your docs there is always a nextcloud subfolder. I am running Nextcloud directly with my own domain.

Rello · December 11, 2019, 8:02pm

hello,
the only think I can find is the “,” too much at the end.
I am not sure if this is the issue because it does not cause any errors on my side.
will patch it with the next release

github.com

Rello/analytics/blob/master/lib/Controller/ApiDataController.php#L49


    ILogger $logger,
    IUserSession $userSession,
    ActivityManager $ActivityManager,
    DatasetController $DatasetController,
    StorageController $StorageController
)
{
    parent::__construct(
        $appName,
        $request,
        'POST',
        );
    $this->logger = $logger;
    $this->userSession = $userSession;
    $this->ActivityManager = $ActivityManager;
    $this->DatasetController = $DatasetController;
    $this->StorageController = $StorageController;
}


/**
 * @CORS

which NC Version?

zx81 · December 15, 2019, 10:48am

After upgrade to 1.2.0 the table data now appears empty, I mean, the chart is ok, the table does not show data, no matter if I choose chart+table or only table.

Rello · December 15, 2019, 11:18am

Rello · December 15, 2019, 2:42pm

fixed with 1.2.2

jakobssystems · December 16, 2019, 9:03am

Hi Rello, just for you as feedback, after current updates everything runs as intended.

isi · March 3, 2020, 10:19am

Dear all and @Rello,

I am currently looking for collaborators to develop an advanced data visualization app within Nextcloud. The aim is to collect metadata from different resources like social media, website tracks (via matomo e.g.), ticketing data (which could be uploaded as “offline data” as an excel file). This is currently do-able with Google Tag Manager, Google Data Studio and services like Supermetrics.

The overall aim is to provide an easy to use app to help micro-sized cultural organisations to learn from their “owned data” and improve their events and festivals while providing a safe place for their data. The project includes the development of a manual on how to set this up for non-techs.

Would that be something someone would be interested in collaborating on?

Rello · March 3, 2020, 11:16am

Hello @isi
Thank you for the feedback. Exactly this is the idea behind Data Analytics and its roadmap. Provide a - as far as possible - flexible way to store and report your own data.

The ETL process is based on different datasources. E.g. csv files or json (upcoming release). And there is an Rest API to push data into DA.

Your cases are valid - but also very special an I am not sure this is a good way to hard code everything. Ways like pushing data via Alerxy or IFTT into DA will always be the most flexible in my opinion.

The other way if you want to deliver processes out of the box: datasources via addons to DA. The App is prepared (not fully ready yet) to be flexible with datasources. So a dev could implement his own datasource and publish this as addon in the appstore. This ensured a disconnection between the main app and its content delivery

Do you think this could be a way to evaluate further? I would love to look into supporting the first DA-Addon for e.g. Facebook data

Cheers
Marcel