Looking forward to develop "Sorts", files visualisation improvement and filters

[Aeredren]

Hello there,

Following a poll-based study on Nextcloud users in social activists communities, we at Framasoft (french non-profit) identify some struggles from this user base.

Results will be published soon, and among those was the UX of “Files”. As an instance, some users reports to always going back to “home” because they do not know how to find their ways in Files. Others reports difficulties understanding how their organization was sorting things, resulting in everyone having his owns sorting patterns with no coherence whatsoever.

We have decided that I’ll try to address this issue, and we came up with “Nextcloud Sorts”.

The idea behind this standalone app is to provide a new interface for visualizing the tree of files and folders. Because a list worth 100 paragraphs, here is the scope of this app :

• Provide a better “List view” …
  • … with the ability to unroll multiple folders instead of the actual behaviour of diving into folders.
  • … with more information accessible from the main view (especially Tags)
• A (de-unified) search bar which search exclusively into files and folders, by name. (something like this draft : #29614)
• User defined filters (where you can search for things like : all the files tagged “urgent” which are more than 500MB)
  • Integration of already existing filters (shared, favorites, and tags)

It can also be seen as a proof of concept for what a future “Files” interface in Vue could look like.

And here is what is not in scope :

• Automatic tagging
• Replacing the Files app
  • The scope end up at visualizing things
  • no file modification
  • no file movement
  • no file sharing interface.

A button to open files in “Files” will be provided instead.

The rational is : if I messed up file visualization, my app is just useless whereas if I messed up data modification my app is harmful. Furthermore, I only have 3 month of internship left to develop this, hence a limited scope.

I’ll put a link to a git repository once I’ll have something to showcase. For now I have just made this prototype. Please note : this is just a mock-up. Also, click on “Interactions → show interaction” to see clickable elements (top right-hand corner).

Below, 2 screenshots from the mockup :

-.

I do not have any mock-up for mobile interface yet, even if I should have started with it (“mobiles first”). But I have some clues on how to do it and a mobile mockup might be available soon. (We are talking mobile web browsers here, not mobile apps)

Any reactions and critics are welcome, and if you feel like we are re-inventing the wheel, please do tell us.

RLT (Aeredren),
Intern at Framasoft.
romain@framasoft.org

[tflidd]

@jan do you know if Nextcloud has something in the pipeline?

[Daphne]

Hi from Nextcloud! I wanted to thank you for your elaborate and constructive feedback and let you know that your topic has been forwarded to our team. I’m not from the design team myself so won’t be able to give you any more details but just wanted to let you know this didn’t go unnoticed.

[carlschwan]

Salut

This is cool. You might want to take a look at Files to vue · Issue #28063 · nextcloud/server · GitHub

[jan]

@Aeredren

Results will be published soon, and among those was the UX of “Files”. As an instance, some users reports to always going back to “home” because they do not know how to find their ways in Files. Others reports difficulties understanding how their organization was sorting things, resulting in everyone having his owns sorting patterns with no coherence whatsoever.

Hi @Aeredren, Jan from the design team here – thanks for letting us know! Are the results/data of the survey published already by any chance? I’d be very interested in giving it a read.

On your individual points:

@Aeredren

the ability to unroll multiple folders instead of the actual behaviour of diving into folders.

Was this tree view something which was requested? How would it deal with large folders, and deep folder structures?

@Aeredren

more information accessible from the main view (especially Tags)

This is something which could be added to core itself – if someone is using tags, it would be useful to show them e.g. either next to the filename, or in a subline. Check how we do it in Mail, possibly that’s helpful: https://github.com/nextcloud/mail/pull/4894#issuecomment-838246664

@Aeredren

A (de-unified) search bar which search exclusively into files and folders, by name. (something like this draft : #29614)

This would be quite useful too to work on in collaboration with core / Files. cc @skjnldsv @jospoortvliet

@Aeredren

User defined filters (where you can search for things like : all the files tagged “urgent” which are more than 500MB)

For this, I’d recommend developing a mechanism like “recent searches” / “frequent searches”. This is something commonly known from other platforms, and it doesn’t require people to consciously create the filters that they use often, but rather the system remembers.

In general, in would be nice to collaborate on some of these things, and for others there could be apps which enhance the Files capability. This might be better than developing a separate app which needs to be maintained?

[Aeredren]

Hi @jan,
Thank you for your interest on this project.

@jan

Are the results/data of the survey published already by any chance? I’d be very interested in giving it a read.

Not yet. But we could give you raw anonymized data if you want. (But it is all in french). For published results, I am not the one working on it but we let you know once we publish something.

@jan

Was this tree view something which was requested? How would it deal with large folders, and deep folder structures?

Yes it was requested, sometime explicitly and sometime it was just implicit (people complaining about struggle to understand the nested folder hierarchy of their association / company / …)

For large folders we could use paging with a limit of, lets say, 20 nodes displayed per folder and a “more” button to fetch 20 more nodes.

For deep folder structure, the UI will inevitably grow larger and larger. We could add an horizontal scroll but it might not be necessary : I think the common usage when using “tree like” file browsers is to change the “root folder” from which the tree is display when the number of nested folders displayed is too high. For now I plan on doing it this way :

• simple click = “unroll” the folder
• double click = “enter” the folder

I feel like it is quite the “standard behaviour” among file browser but it is just a feeling.

@jan

For this, I’d recommend developing a mechanism like “recent searches” / “frequent searches”. This is something commonly known from other platforms, and it doesn’t require people to consciously create the filters that they use often, but rather the system remembers.

Agreed. It was more or less the idea behind what I called “Quickfilters” which are more likes “favorites for user-defined filters”.

@jan

In general, in would be nice to collaborate on some of these things, and for others there could be apps which enhance the Files capability. This might be better than developing a separate app which needs to be maintained?

I agree, “unified search” and “displaying tags” should be upstream features, and having yet another app to maintain is a bad thing.

The main problem I’ll face with your proposition is : I am short on time.

My internship end at the end of February, and as core / files developers plans on rewriting “files” front-end in Vuejs we thought the contribution process to “files” could be hazardous.

Therefore our intention were : “I’ll explore the feasibility of ‘tree view’ and ‘conditional filtering’” with this app being more like a Proof Of Concept. (Framasoft have no intention to further develop this app once I’ll be on my way. Just to do basic maintenance as the app could still be useful for users who need conditional filters)

But all in all, if my code is somewhat re-usable in upstream, and if time allow it, I’ll be glad to contribute to already existing core/files.

[StCyr]

@Aeredren

A (de-unified) search bar which search exclusively into files and folders, by name. (something like this draft : #29614)

Will this search bar be able to search recursively in subfolders?

This is a missing functionality for large folders (ie: containing a lot of subfolders) for which I’ve developed an app (Cyrille Bollu / Nextcloud advanced search · GitLab). I would gladly share my code with you if you like

[Aeredren]

@StCyr

Will this search bar be able to search recursively in subfolders?

Yes indeed, it should search recursively through all users files. Is it not the standard files search behaviour ?

@StCyr

This is a missing functionality for large folders (ie: containing a lot of subfolders) for which I’ve developed an app (https://framagit.org/StCyr/files_advancedsearch ). I would gladly share my code with you if you like

I took a look at it and it seem indeed quite similar for searching through various files attribute, but I’d like to have the SystemTags queries managed by the backend for pagination, and negations manage by the backend too in order to reduce DB

Merci beaucoup for sharing this anyways, could come in handy.

I’ll publish my git repositories soon, but I want to have something usable first.

[StCyr]

@Aeredren

Yes indeed, it should search recursively through all users files. Is it not the standard files search behaviour ?

I meant “search recursively within the current folder”.

The current behavior of the unified search is to search recursively through all users files no matter where you are in your file’s hierarchy.

So, let’s say you have the following file’s hierarchy and you are looking for a file in folderB (and its subfolders) only (ie: you know the file you are looking for is somewhere in that folderB or one of its subfolders):

/— folderA (big folder with 1000+ files)
|
|—folderB (big folder with 1000+ files)
| |
| |----subfolderB1 (folder with ~100 files)
|
| |----subfolderB2(folder with ~300 files)
|
| |----subfolderB3 (folder with ~500 files)
|
| |----subfolderB4 (folder with ~100 files)
|
|—folderC (big folder with 1000+ files)
|
|—folderD (big folder with 1000+ files)

It’s currently not possible to search within folderB and its subfolders only: NC’s unified search 'scope is the 4000+ files of your root folder, not the 1000+ files of folderB. I believe it’s a serious limitation of NC’s unified search

[Aeredren]

@StCyr

I meant “search recursively within the current folder”.

Okk, I did’nt get it sorry, but I get it now.

I hadn’t thought of it as default behaviour but it could be nice. I’ll give it some thought.

[StCyr]

I felt that this was the right time to raise this point since @jan said “This would be quite useful too to work on in collaboration with core / Files. cc @skjnldsv @jospoortvliet”

[lironb]

Hi @Aeredren!
This is a feature I was looking for lately and I think your proposal & mock-up are great.
I also think that search/filter history and recursive search inside a specific directory will be very useful.
Do you think it will also be easy to add these options to the search?

1. File creator/modifier
2. Comments - who commented, comment text
3. File type
4. Full-text search

Thanks!

[Aeredren]

@lironb
As I did it, I use a lot the search functionnality from OCP\Files\Folder which allow filters through those category

$types = [
	'mimetype' => 'string',
	'mtime' => 'integer',
	'name' => 'string',
	'path' => 'string',
	'size' => 'integer',
	'tagname' => 'string', // this are not systemTag, but old tags only used by 'favorite' now (this field is now pretty useless)
	'favorite' => 'boolean',
	'fileid' => 'integer',
	'storage' => 'integer',
];

Anything other than that might be possible, but it will cost efficiency. Combining any of the search above always result in one SQL request. Adding other search parameter will mean other requests or php treatment will be made in some kind of ugly hack… I already plan on doing it for systemTags (tags) and Shares information, one might build upon that to add others.

So to answer:

1. File creator/modifier not easy
2. Comments - who commented, comment text not easy
3. File type easy
4. Full-text search not easy

Thanks for this proposition

[lironb]

@Aeredren Thanks for the fast and informative reply
It will be great if you can share your project’s code once you have some working POC with systemTags/Shares.
WIll keep watching this post for any progress.
Thanks,
Liron

[jospoortvliet]

Would love to hear if there’s been any progress!

[jospoortvliet]

… or maybe now

[Aeredren]

Hi

There’s been some progress indeed !
I finished my internship one month ago with a “sorts app” good enough for a beta version. I started the publication process at the end of february / begining of march, but I had no time to complete it, until now. I will be working on it on my free time this week-end and next week

An overview of what we achieve with the original scope:

@Aeredren

• Provide a better “List view” …
  • … with the ability to unroll multiple folders instead of the actual behaviour of diving into folders.
  • … with more information accessible from the main view (especially Tags)

Done

@Aeredren

• A (de-unified) search bar which search exclusively into files and folders, by name. (something like this draft : #29614)

It’s not a search bar, but a filter on “name” behave the same, so I consider it done

@Aeredren

• User defined filters (where you can search for things like : all the files tagged “urgent” which are more than 500MB)
  • Integration of already existing filters (shared, favorites, and tags)

Also done but with some drawbacks and possible improvements: It might be to heavy for big instances due to the way I combined some filters, and I did not finished filters on shares.

Code is available at Framasoft / Nextcloud / Sorts · GitLab

I’ll keep this thread inform of any progress during the next few weeks

RLT

[just]

Awesome! We have a group of volunteer testers who will be happy to help with testing the Sorts app. @bug-hunters

[Aeredren]

Hello there,

I’m back working on this App after a 4 month gaps to focus on my personal life and other project.

We, at Framasoft, finally publish Sorts on Nextcloud Store !

The plugin now enter a period of volunteer development and although I will still watch gitlab issues and merge requests, further development will be done on my free time and no deadline are to be expected.

I’ll look toward adding search on shares metadata as it is a major aspect of Nextcoud, fixing main issues if any and publish Sorts for new Nextcloud versions.

We plan to communicate on the release on Framasoft’s blog, in french mostly, and will see if it grasp the interest of the community. (As the main communication canal so far was this thread, Sorts didn’t get a lot of attention (yet ?) )

@just , with the now freshly released beta, if anyone is still interested in testing the app, I’ll gladly take into account any feedback !

~Aeredren

[pyg]

@all : we’re please to announce Sorts publicly :slight_smile

Please read Nextcloud "Sorts" release 0.0.1: an app for files tree navigation and advanced files filters

Full article and announce in (approximate) english here: Nextcloud Sorts: a Nextcloud application prototype to navigate your files more easily – Framablog