OwnClouds Future

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

cody@killsudo.info
Could OwnCloud be used to seperate kde and the home folder from any physical
computer.

I would one day like to be able to keep my kde desktop/resources/data in the
cloud and simple use a domain login to request and cache my desktop to any
kde4 pc or device. I imagine kdm supporting a login so I could specifiy
cody at killsudo.info and it would connect to my owncloud instance on the server
and load my desktop and files. My widgets, themes, mail, tags,  etc would all
be pulled down.

I would even like to push akonadi to the server and connect my kde apps up to
the cloud to store and send/receive data.

A phone running kde(or any app that uses open-standards) could connect to my
same instance and using akondi pull my mail, contacts, etc. Could even stream
music on the go from my owncloud instance to the phone.

Internet speeds are getting fast enough even wirelessly to make this a final
reality.

Right now I use libgcal and akonadi resources to plan my calendar then push
the calendar to my google calendar that can then be shared with family.

I would also like to see a nice web frontend over owncloud as you are doing
now that lets me access my data, contacts, etc in a browser if need be to make
quick changes or share a file.

So, am I crazy?

Signed

Cody

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Mario Đanić
Hi Cody,

please keep in mind that I have not contributed a single line of code
to OwnCloud ... yet :)

The ideas you have certainly sound feasible, at least from the
technical point of view.
You would have to come up with a format in which you would exchange information,
pretty much like Tomboy folks did for notes ... hopefully that will go
into OwnCloud as a
plugin one day as well.

As you probably already know, it would also require changes to various
desktop elements.
Not impossible, but requires collaboration from a lot of people.

Crazy? Hardly. But I don't think it will happen in the short term.

Cheers,
Mario

On Sat, Mar 20, 2010 at 23:48,  <cody at killsudo.info> wrote:

> Could OwnCloud be used to seperate kde and the home folder from any physical
> computer.
>
> I would one day like to be able to keep my kde desktop/resources/data in the
> cloud and simple use a domain login to request and cache my desktop to any
> kde4 pc or device. I imagine kdm supporting a login so I could specifiy
> cody at killsudo.info and it would connect to my owncloud instance on the server
> and load my desktop and files. My widgets, themes, mail, tags, ?etc would all
> be pulled down.
>
> I would even like to push akonadi to the server and connect my kde apps up to
> the cloud to store and send/receive data.
>
> A phone running kde(or any app that uses open-standards) could connect to my
> same instance and using akondi pull my mail, contacts, etc. Could even stream
> music on the go from my owncloud instance to the phone.
>
> Internet speeds are getting fast enough even wirelessly to make this a final
> reality.
>
> Right now I use libgcal and akonadi resources to plan my calendar then push
> the calendar to my google calendar that can then be shared with family.
>
> I would also like to see a nice web frontend over owncloud as you are doing
> now that lets me access my data, contacts, etc in a browser if need be to make
> quick changes or share a file.
>
> So, am I crazy?
>
> Signed
>
> Cody
> _______________________________________________
> Owncloud mailing list
> Owncloud at kde.org
> https://mail.kde.org/mailman/listinfo/owncloud
>

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Cosmin Gorgovan
In reply to this post by cody@killsudo.info
I think it should be possible to develop an webdav fuse driver optimized
for this kind of use;
and to modify KDM to 1) allow this kind of logins and 2)mount your
owncloud as your home directory after logging in

This would result in having the same dot-files on every computer you log
in on, so you'd basically get what you want (and probably get in some
trouble if you are running significantly different versions of the
apps). With decent optimisations, I think this would work quite OK,
maybe except for the initial sync where you'd be limited by the
bandwidth, transfer overhead and number of files.

Unless someone else sees a fatal flaw, I might just turn this into my
GSoC proposal for OwnCloud.

Cheers,
Cosmin


On 03/20/2010 10:48 PM, cody at killsudo.info wrote:

> Could OwnCloud be used to seperate kde and the home folder from any physical
> computer.
>
> I would one day like to be able to keep my kde desktop/resources/data in the
> cloud and simple use a domain login to request and cache my desktop to any
> kde4 pc or device. I imagine kdm supporting a login so I could specifiy
> cody at killsudo.info and it would connect to my owncloud instance on the server
> and load my desktop and files. My widgets, themes, mail, tags,  etc would all
> be pulled down.
>
> I would even like to push akonadi to the server and connect my kde apps up to
> the cloud to store and send/receive data.
>
> A phone running kde(or any app that uses open-standards) could connect to my
> same instance and using akondi pull my mail, contacts, etc. Could even stream
> music on the go from my owncloud instance to the phone.
>
> Internet speeds are getting fast enough even wirelessly to make this a final
> reality.
>
> Right now I use libgcal and akonadi resources to plan my calendar then push
> the calendar to my google calendar that can then be shared with family.
>
> I would also like to see a nice web frontend over owncloud as you are doing
> now that lets me access my data, contacts, etc in a browser if need be to make
> quick changes or share a file.
>
> So, am I crazy?
>
> Signed
>
> Cody
> _______________________________________________
> Owncloud mailing list
> Owncloud at kde.org
> https://mail.kde.org/mailman/listinfo/owncloud
>    


Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Mario Đanić
I see one flaw - for this you need desktop implementation of OwnCloud
in a form of a
daemon and an application for syncing.

I plan on working on this (daemon and sync app), either for KDE or
GNOME organizations, but it would be great if you would like to
collaborate as our projects overlap - perhaps work together on the
backend, I'll work on the UI, and you can modify the apps to make it
work with OwnCloud.
I am also inviting the other guy that wishes to do UI for other DE, to
collaborate. Reusing effort and collaborating is good. The backend and
the UI would be written in python.

That being said, the approach you mentioned is a very rough one. I
have some ideas
in mind that might improve situation, namely:

- collect basic info about the computer, and attach it to particular
home folder in OwnCloud
- if you want to mount it on another computer, inspect that computer
and see if there are
 any potential problems
- if there are no problem, just mount it, otherwise say there might be
potential conflicts

That being said, I'd say we should avoid syncing dot-files for now:
its not easy to get
it right, and you still get most of the functionality. However I can
see that this has some
shortcomings, especially in applications like firefox that holds
profile data in such
files and directories, so we *might* selectively choose that.

I must say that I am really happy to see so much interest in OwnCloud. Let's see
what the future holds for us.

Cheers,
Mario

On Sun, Mar 21, 2010 at 05:18, Cosmin Gorgovan <cosmin at linux-geek.org> wrote:

> I think it should be possible to develop an webdav fuse driver optimized
> for this kind of use;
> and to modify KDM to 1) allow this kind of logins and 2)mount your
> owncloud as your home directory after logging in
>
> This would result in having the same dot-files on every computer you log
> in on, so you'd basically get what you want (and probably get in some
> trouble if you are running significantly different versions of the
> apps). With decent optimisations, I think this would work quite OK,
> maybe except for the initial sync where you'd be limited by the
> bandwidth, transfer overhead and number of files.
>
> Unless someone else sees a fatal flaw, I might just turn this into my
> GSoC proposal for OwnCloud.
>
> Cheers,
> Cosmin
>
>
> On 03/20/2010 10:48 PM, cody at killsudo.info wrote:
>> Could OwnCloud be used to seperate kde and the home folder from any physical
>> computer.
>>
>> I would one day like to be able to keep my kde desktop/resources/data in the
>> cloud and simple use a domain login to request and cache my desktop to any
>> kde4 pc or device. I imagine kdm supporting a login so I could specifiy
>> cody at killsudo.info and it would connect to my owncloud instance on the server
>> and load my desktop and files. My widgets, themes, mail, tags, ?etc would all
>> be pulled down.
>>
>> I would even like to push akonadi to the server and connect my kde apps up to
>> the cloud to store and send/receive data.
>>
>> A phone running kde(or any app that uses open-standards) could connect to my
>> same instance and using akondi pull my mail, contacts, etc. Could even stream
>> music on the go from my owncloud instance to the phone.
>>
>> Internet speeds are getting fast enough even wirelessly to make this a final
>> reality.
>>
>> Right now I use libgcal and akonadi resources to plan my calendar then push
>> the calendar to my google calendar that can then be shared with family.
>>
>> I would also like to see a nice web frontend over owncloud as you are doing
>> now that lets me access my data, contacts, etc in a browser if need be to make
>> quick changes or share a file.
>>
>> So, am I crazy?
>>
>> Signed
>>
>> Cody
>> _______________________________________________
>> Owncloud mailing list
>> Owncloud at kde.org
>> https://mail.kde.org/mailman/listinfo/owncloud
>>
>
> _______________________________________________
> Owncloud mailing list
> Owncloud at kde.org
> https://mail.kde.org/mailman/listinfo/owncloud
>

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Cosmin Gorgovan
Hi,

> I see one flaw - for this you need desktop implementation of OwnCloud
> in a form of a
> daemon and an application for syncing.
>    
Yeah, I was proposing an implementation using FUSE because this way you
can make applications wait until a file becomes available before
allowing them access to it. This would prevent race conditions like
starting firefox, firefox creating a new profile because the existent
one hasn't been downloaded yet and then being unable to download the
original file because the new one is locked.
I consider this essential if you want more functionality than dropbox
(eg. using owncloud for your whole home directory).
> I plan on working on this (daemon and sync app), either for KDE or
> GNOME organizations, but it would be great if you would like to
> collaborate as our projects overlap - perhaps work together on the
> backend, I'll work on the UI, and you can modify the apps to make it
> work with OwnCloud.
>    
I think it would be orders of magnitude easier and faster to do decent
file syncing (that is, not too slow) for dot files instead of modifying
every application out there to use a custom library for saving settings.

The other thing is that at least for the moment I intend to work through
GSoC, and as this is a KDE project, I suppose my work would need to be
mostly DE independent or KDE-specific. By the way, are you also a GSoC
student?
> I am also inviting the other guy that wishes to do UI for other DE, to
> collaborate. Reusing effort and collaborating is good.
Sure. The only thing to keep in mind is that in the context of GSoC
there's not too much time, so the tasks would need to be fairly discrete.
> The backend and
> the UI would be written in python.
>    
I was also going to suggest Python, with the mention that I'd like to
test the disk and network I/O speed more thoroughly.
> That being said, the approach you mentioned is a very rough one. I
> have some ideas
> in mind that might improve situation, namely:
>
> - collect basic info about the computer, and attach it to particular
> home folder in OwnCloud
>    
That sounds like something to use when otherwise things start breaking.
> - if you want to mount it on another computer, inspect that computer
> and see if there are
>   any potential problems
>    
Those being file conflicts?

> - if there are no problem, just mount it, otherwise say there might be
> potential conflicts
>
> That being said, I'd say we should avoid syncing dot-files for now:
> its not easy to get
> it right, and you still get most of the functionality. However I can
> see that this has some
> shortcomings, especially in applications like firefox that holds
> profile data in such
> files and directories, so we *might* selectively choose that.
>    
Almost all applications save user data in dot files. Personally, I have
switched between different distributions and kept my home directory
intact for a number of times without encountering any significant
problem. What we'd need is an option for creating symlinks, because
files used by the same application might have slightly different names
on different distributions.
> I must say that I am really happy to see so much interest in OwnCloud. Let's see
> what the future holds for us.
>
> Cheers,
> Mario
>    
Exciting times ahead :)

Thanks,
Cosmin

> On Sun, Mar 21, 2010 at 05:18, Cosmin Gorgovan<cosmin at linux-geek.org>  wrote:
>    
>> I think it should be possible to develop an webdav fuse driver optimized
>> for this kind of use;
>> and to modify KDM to 1) allow this kind of logins and 2)mount your
>> owncloud as your home directory after logging in
>>
>> This would result in having the same dot-files on every computer you log
>> in on, so you'd basically get what you want (and probably get in some
>> trouble if you are running significantly different versions of the
>> apps). With decent optimisations, I think this would work quite OK,
>> maybe except for the initial sync where you'd be limited by the
>> bandwidth, transfer overhead and number of files.
>>
>> Unless someone else sees a fatal flaw, I might just turn this into my
>> GSoC proposal for OwnCloud.
>>
>> Cheers,
>> Cosmin
>>
>>
>> On 03/20/2010 10:48 PM, cody at killsudo.info wrote:
>>      
>>> Could OwnCloud be used to seperate kde and the home folder from any physical
>>> computer.
>>>
>>> I would one day like to be able to keep my kde desktop/resources/data in the
>>> cloud and simple use a domain login to request and cache my desktop to any
>>> kde4 pc or device. I imagine kdm supporting a login so I could specifiy
>>> cody at killsudo.info and it would connect to my owncloud instance on the server
>>> and load my desktop and files. My widgets, themes, mail, tags,  etc would all
>>> be pulled down.
>>>
>>> I would even like to push akonadi to the server and connect my kde apps up to
>>> the cloud to store and send/receive data.
>>>
>>> A phone running kde(or any app that uses open-standards) could connect to my
>>> same instance and using akondi pull my mail, contacts, etc. Could even stream
>>> music on the go from my owncloud instance to the phone.
>>>
>>> Internet speeds are getting fast enough even wirelessly to make this a final
>>> reality.
>>>
>>> Right now I use libgcal and akonadi resources to plan my calendar then push
>>> the calendar to my google calendar that can then be shared with family.
>>>
>>> I would also like to see a nice web frontend over owncloud as you are doing
>>> now that lets me access my data, contacts, etc in a browser if need be to make
>>> quick changes or share a file.
>>>
>>> So, am I crazy?
>>>
>>> Signed
>>>
>>> Cody
>>> _______________________________________________
>>> Owncloud mailing list
>>> Owncloud at kde.org
>>> https://mail.kde.org/mailman/listinfo/owncloud
>>>
>>>        
>> _______________________________________________
>> Owncloud mailing list
>> Owncloud at kde.org
>> https://mail.kde.org/mailman/listinfo/owncloud
>>
>>      


Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Mario Đanić
Hi,

please find my comments in-lined.

On Sun, Mar 21, 2010 at 22:10, Cosmin Gorgovan <cosmin at linux-geek.org> wrote:

> Hi,
>
>> I see one flaw - for this you need desktop implementation of OwnCloud
>> in a form of a
>> daemon and an application for syncing.
>>
>
> Yeah, I was proposing an implementation using FUSE because this way you can
> make applications wait until a file becomes available before allowing them
> access to it. This would prevent race conditions like starting firefox,
> firefox creating a new profile because the existent one hasn't been
> downloaded yet and then being unable to download the original file because
> the new one is locked.
> I consider this essential if you want more functionality than dropbox (eg.
> using owncloud for your whole home directory).
>>

Doesn't FUSE use a lot of memory while you're on battery? Hm...

>> I plan on working on this (daemon and sync app), either for KDE or
>> GNOME organizations, but it would be great if you would like to
>> collaborate as our projects overlap - perhaps work together on the
>> backend, I'll work on the UI, and you can modify the apps to make it
>> work with OwnCloud.
>>
>
> I think it would be orders of magnitude easier and faster to do decent file
> syncing (that is, not too slow) for dot files instead of modifying every
> application out there to use a custom library for saving settings.
>

Actually, what I meant are applications like KDM or/and GMD.

> The other thing is that at least for the moment I intend to work through
> GSoC, and as this is a KDE project, I suppose my work would need to be
> mostly DE independent or KDE-specific. By the way, are you also a GSoC
> student?

I always vote for DE independent. That's why my sync client architecture was
designed with that in mind - backend separated from the UIs.
>>
>> I am also inviting the other guy that wishes to do UI for other DE, to
>> collaborate. Reusing effort and collaborating is good.
>
> Sure. The only thing to keep in mind is that in the context of GSoC there's
> not too much time, so the tasks would need to be fairly discrete.
>>

Of course.
>> The backend and
>> the UI would be written in python.
>>
>
> I was also going to suggest Python, with the mention that I'd like to test
> the disk and network I/O speed more thoroughly.
>>

For critical tasks, we could use C-extensions (for example, rsync).

>> That being said, the approach you mentioned is a very rough one. I
>> have some ideas
>> in mind that might improve situation, namely:
>>
>> - collect basic info about the computer, and attach it to particular
>> home folder in OwnCloud
>>
>
> That sounds like something to use when otherwise things start breaking.
>>
>> - if you want to mount it on another computer, inspect that computer
>> and see if there are
>> ?any potential problems
>>
>
> Those being file conflicts?

And others - potential config file -> installed program(s) conflict.

>>
>> - if there are no problem, just mount it, otherwise say there might be
>> potential conflicts
>>
>> That being said, I'd say we should avoid syncing dot-files for now:
>> its not easy to get
>> it right, and you still get most of the functionality. However I can
>> see that this has some
>> shortcomings, especially in applications like firefox that holds
>> profile data in such
>> files and directories, so we *might* selectively choose that.
>>
>
> Almost all applications save user data in dot files. Personally, I have
> switched between different distributions and kept my home directory intact
> for a number of times without encountering any significant problem. What
> we'd need is an option for creating symlinks, because files used by the same
> application might have slightly different names on different distributions.
>>

Somtimes you encounter problems, sometimes you don't. But it adds to the
bloat for sure.

>> I must say that I am really happy to see so much interest in OwnCloud.
>> Let's see
>> what the future holds for us.
>>
>> Cheers,
>> Mario
>>
>
> Exciting times ahead :)
>
Indeed :)
> Thanks,
> Cosmin
>>

Cheers,
Mario

>> On Sun, Mar 21, 2010 at 05:18, Cosmin Gorgovan<cosmin at linux-geek.org>
>> ?wrote:
>>
>>>
>>> I think it should be possible to develop an webdav fuse driver optimized
>>> for this kind of use;
>>> and to modify KDM to 1) allow this kind of logins and 2)mount your
>>> owncloud as your home directory after logging in
>>>
>>> This would result in having the same dot-files on every computer you log
>>> in on, so you'd basically get what you want (and probably get in some
>>> trouble if you are running significantly different versions of the
>>> apps). With decent optimisations, I think this would work quite OK,
>>> maybe except for the initial sync where you'd be limited by the
>>> bandwidth, transfer overhead and number of files.
>>>
>>> Unless someone else sees a fatal flaw, I might just turn this into my
>>> GSoC proposal for OwnCloud.
>>>
>>> Cheers,
>>> Cosmin
>>>
>>>
>>> On 03/20/2010 10:48 PM, cody at killsudo.info wrote:
>>>
>>>>
>>>> Could OwnCloud be used to seperate kde and the home folder from any
>>>> physical
>>>> computer.
>>>>
>>>> I would one day like to be able to keep my kde desktop/resources/data in
>>>> the
>>>> cloud and simple use a domain login to request and cache my desktop to
>>>> any
>>>> kde4 pc or device. I imagine kdm supporting a login so I could specifiy
>>>> cody at killsudo.info and it would connect to my owncloud instance on the
>>>> server
>>>> and load my desktop and files. My widgets, themes, mail, tags, ?etc
>>>> would all
>>>> be pulled down.
>>>>
>>>> I would even like to push akonadi to the server and connect my kde apps
>>>> up to
>>>> the cloud to store and send/receive data.
>>>>
>>>> A phone running kde(or any app that uses open-standards) could connect
>>>> to my
>>>> same instance and using akondi pull my mail, contacts, etc. Could even
>>>> stream
>>>> music on the go from my owncloud instance to the phone.
>>>>
>>>> Internet speeds are getting fast enough even wirelessly to make this a
>>>> final
>>>> reality.
>>>>
>>>> Right now I use libgcal and akonadi resources to plan my calendar then
>>>> push
>>>> the calendar to my google calendar that can then be shared with family.
>>>>
>>>> I would also like to see a nice web frontend over owncloud as you are
>>>> doing
>>>> now that lets me access my data, contacts, etc in a browser if need be
>>>> to make
>>>> quick changes or share a file.
>>>>
>>>> So, am I crazy?
>>>>
>>>> Signed
>>>>
>>>> Cody
>>>> _______________________________________________
>>>> Owncloud mailing list
>>>> Owncloud at kde.org
>>>> https://mail.kde.org/mailman/listinfo/owncloud
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Owncloud mailing list
>>> Owncloud at kde.org
>>> https://mail.kde.org/mailman/listinfo/owncloud
>>>
>>>
>
>

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Mario Đanić
In reply to this post by Cosmin Gorgovan
Oh and yes, this year I am a potential GSoC student.

Cheers,
Mario

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Mario Đanić
Just found this, no idea if it'll be of any use:
http://savannah.nongnu.org/projects/davfs2

Cheers,
Mario

On Sun, Mar 21, 2010 at 22:15, Mario ?ani? <mario.danic at gmail.com> wrote:
> Oh and yes, this year I am a potential GSoC student.
>
> Cheers,
> Mario
>

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Cosmin Gorgovan
On 03/21/2010 09:17 PM, Mario ?ani? wrote:
> Just found this, no idea if it'll be of any use:
> http://savannah.nongnu.org/projects/davfs2
>    
I've tried using it and it looks like it's trying to do some types of
requests not supported by ownCloud yet.
Nautilus instead can mount an ownCloud share, but it creates
case-sensitive file names and displays everything upper case, which I
found weird. I suppose WebDAV shares might be case insensitive.

In any case, I want to implement extensive caching, read-ahead and other
optimizations.

> Cheers,
> Mario
>
> On Sun, Mar 21, 2010 at 22:15, Mario ?ani?<mario.danic at gmail.com>  wrote:
>    
>> Oh and yes, this year I am a potential GSoC student.
>>
>> Cheers,
>> Mario
>>
>>      


Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Mario Đanić
And I have interesting in making things work across platforms and DEs. :)

Cheers,
Mario

On Sun, Mar 21, 2010 at 22:28, Cosmin Gorgovan <cosmin at linux-geek.org> wrote:

> On 03/21/2010 09:17 PM, Mario ?ani? wrote:
>>
>> Just found this, no idea if it'll be of any use:
>> http://savannah.nongnu.org/projects/davfs2
>>
>
> I've tried using it and it looks like it's trying to do some types of
> requests not supported by ownCloud yet.
> Nautilus instead can mount an ownCloud share, but it creates case-sensitive
> file names and displays everything upper case, which I found weird. I
> suppose WebDAV shares might be case insensitive.
>
> In any case, I want to implement extensive caching, read-ahead and other
> optimizations.
>>
>> Cheers,
>> Mario
>>
>> On Sun, Mar 21, 2010 at 22:15, Mario ?ani?<mario.danic at gmail.com> ?wrote:
>>
>>>
>>> Oh and yes, this year I am a potential GSoC student.
>>>
>>> Cheers,
>>> Mario
>>>
>>>
>
>

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Cosmin Gorgovan
In reply to this post by Cosmin Gorgovan
Hello,
> On Mon, Mar 22, 2010 at 12:28 AM, Cosmin Gorgovan<cosmin at linux-geek.org>  wrote:
>    
>> In any case, I want to implement extensive caching, read-ahead and other
>> optimizations.
>>      
> These things must be done by very low-level stuff provided by
> operating system, e.g. davfs might use either its own caching or put
> it on VFS and driver (afair in Linux most caching is done by VFS and
> driver (network, disk, etc)).
My proposal was to write a FUSE (that is, file system in userspace)
driver because:

    * davfs2 doesn't work (at least at the moment) with ownCloud
    * you need root privileges to mount a share
    * from what I've read, it doesn't do read-ahead

And if using a regular daemon for synchronizing your home directory
you'd probably get lots of conflicts because you won't be able to tell
apps "wait before I check if that file already exists", etc.
>   Syncing client is not system software
>    
I'd say a python script running under the user's account isn't either.
> and it provides operations on high level: versioning,
Shouldn't the server do that, especially as the project features say it
needs to get rid of older revisions if it is running out of space?

>   conflicts
> resolution/resolving, mapping (map files/directories from mountpoint
> to user's files/directories), ?ncryption/decription, and so on
> depending on your fantasy. Also it's independent on davfs - it justs
> perfoms appropriate command to mount something to somewhere and that's
> all in this area. But if it makes coffee for user, it will be great
> feature :)
>
>
>    


Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Evgeniy Ivanov
On Mon, Mar 22, 2010 at 1:34 AM, Cosmin Gorgovan <cosmin at linux-geek.org> wrote:

> Hello,
>>
>> On Mon, Mar 22, 2010 at 12:28 AM, Cosmin Gorgovan<cosmin at linux-geek.org>
>>  wrote:
>>
>>>
>>> In any case, I want to implement extensive caching, read-ahead and other
>>> optimizations.
>>>
>>
>> These things must be done by very low-level stuff provided by
>> operating system, e.g. davfs might use either its own caching or put
>> it on VFS and driver (afair in Linux most caching is done by VFS and
>> driver (network, disk, etc)).
>
> My proposal was to write a FUSE (that is, file system in userspace) driver
> because:

Well, your FUSE driver have to read/write its contents from server in
some way. Here you should start thinking about using any existing
protocol or reinventing the wheel. New protocol requires server-side
application, which is additional work. Old protocols might have
limitations, but also existing software, thus FUSE doesn't look very
attractive (better to implement extensions/improvements for available
software). Also fuse driver is not portable (and KDE is).
Anyway implementing filesystem (userspace or kernelspace) is
complicated task and doubt it might be interesting for KDE project.

>   * davfs2 doesn't work (at least at the moment) with ownCloud

It can become part of your GSoC project.

>   * you need root privileges to mount a share

It should be solvable, since you don't have problems with removable devices.

>   * from what I've read, it doesn't do read-ahead

Well, I'm not sure if it can be done via http. AFAIK you can only
request some file and that's all since server can do path-->file
(data) only. Of course there is a way to get only parts of file using
special convention (but here you fell into trouble with limitations to
directory structure and some other things) and it's done in python
very quickly (but here again you will have server side application).

> And if using a regular daemon for synchronizing your home directory you'd
> probably get lots of conflicts because you won't be able to tell apps "wait
> before I check if that file already exists", etc.
>>
>>  Syncing client is not system software
>>
>
> I'd say a python script running under the user's account isn't either.

Sure, that's what I meant.

>> and it provides operations on high level: versioning,
>
> Shouldn't the server do that, especially as the project features say it
> needs to get rid of older revisions if it is running out of space?

Depends on functionality which client provides, e.g. it should be able
to get requested version from the server, thus it should know about
which versioning mechanism is used.

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Evgeniy Ivanov
On Mon, Mar 22, 2010 at 2:10 AM, Evgeniy Ivanov <lolkaantimat at gmail.com> wrote:
>>   * from what I've read, it doesn't do read-ahead
>
> Well, I'm not sure if it can be done via http. AFAIK you can only
> request some file and that's all since server can do path-->file
> (data) only. Of course there is a way to get only parts of file using
> special convention (but here you fell into trouble with limitations to
> directory structure and some other things) and it's done in python
> very quickly (but here again you will have server side application).
>

Btw, since AFAIK http can provide only full files (not random blocks)
read-ahead can's be done for http-based filesystem, since all you can
is either get full file or get nothing. You don't know what to read
ahead.



--
Evgeniy Ivanov

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Cosmin Gorgovan
On 03/21/2010 11:13 PM, Evgeniy Ivanov wrote:

> On Mon, Mar 22, 2010 at 2:10 AM, Evgeniy Ivanov<lolkaantimat at gmail.com>  wrote:
>    
>>>    * from what I've read, it doesn't do read-ahead
>>>        
>> Well, I'm not sure if it can be done via http. AFAIK you can only
>> request some file and that's all since server can do path-->file
>> (data) only. Of course there is a way to get only parts of file using
>> special convention (but here you fell into trouble with limitations to
>> directory structure and some other things) and it's done in python
>> very quickly (but here again you will have server side application).
>>
>>      
> Btw, since AFAIK http can provide only full files (not random blocks)
>    
HTTP supports getting limited ranges of a file. The server's reponse
code is 206 Partial content in case you want to look into that. Just to
make sure, I'm talking about getting the files from the network share to
the local disk cache, not about the "readahead" technology from the
Linux kernel.
> read-ahead can's be done for http-based filesystem, since all you can
> is either get full file or get nothing. You don't know what to read
> ahead.
>
>    
What I'd do would be to get all the files as long as there is disk space
and there isn't any file activity. The order could be determined by a
simple probabilistic model (that might be a little over the top), file
size, or any other criteria.

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Evgeniy Ivanov
On Mon, Mar 22, 2010 at 2:46 AM, Cosmin Gorgovan <cosmin at linux-geek.org> wrote:

> On 03/21/2010 11:13 PM, Evgeniy Ivanov wrote:
>>
>> On Mon, Mar 22, 2010 at 2:10 AM, Evgeniy Ivanov<lolkaantimat at gmail.com>
>> ?wrote:
>>
>>>>
>>>> ? * from what I've read, it doesn't do read-ahead
>>>>
>>>
>>> Well, I'm not sure if it can be done via http. AFAIK you can only
>>> request some file and that's all since server can do path-->file
>>> (data) only. Of course there is a way to get only parts of file using
>>> special convention (but here you fell into trouble with limitations to
>>> directory structure and some other things) and it's done in python
>>> very quickly (but here again you will have server side application).
>>>
>>>
>>
>> Btw, since AFAIK http can provide only full files (not random blocks)
>>
>
> HTTP supports getting limited ranges of a file. The server's reponse code is
> 206 Partial content in case you want to look into that. Just to make sure,

Didn't know it, thanks.


>> read-ahead can's be done for http-based filesystem, since all you can
>> is either get full file or get nothing. You don't know what to read
>> ahead.
>>
> What I'd do would be to get all the files as long as there is disk space and
> there isn't any file activity.

It's something like locality principle. I don't think using disk space
is good option in this case. Nobody wants application to consume much
memory, disk space or network bandwidth. Just imagine, that our
application took last disk space (sure, quota might be set, but...)
and user needs additional space too. So here we came to something very
complicated (quotas and managing free space, probably deallocation on
demand, caching policies, ...). And it's just about reading. Writing
is more complicated process.

Cosmin, I really enjoy discussion, but in my opinion it's beyond this
project. If you really willing to write user-space filesystem, I would
recommend you to focus on it as separate GSoC project (MINIX 3
organization is a good place for this).


--
Evgeniy Ivanov

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Cosmin Gorgovan
On 03/22/2010 01:02 AM, Evgeniy Ivanov wrote:
> Cosmin, I really enjoy discussion, but in my opinion it's beyond this
> project. If you really willing to write user-space filesystem, I would
> recommend you to focus on it as separate GSoC project (MINIX 3
> organization is a good place for this).
>    
OK, I've thought about it and I think you are right. I've also thought
about other possible projects and I've came up with three smaller ones:

   1. Implementing a interface to allow simple integration of new
      database engines. Making it non-relational DB friendly would be
      cool. Writing at least MySQL and sqlite modules.
   2. Implementing multiple users and shared folders/files. Using WebDAV
      ACL[1] seems the way to go for the communication part. The
      database/database interface I mentioned earlier would require some
      modifications.
   3. Implementing versioning. There's a WebDAV extension[2] for this
      too. The database would require modifications again.

The last two items would require working in close collaboration with the
desktop client developer(s) for agreeing upon the exact implementation
details.

So, what does everyone think?

Thanks,
Cosmin


[1]: http://tools.ietf.org/html/rfc3744
[2]: http://tools.ietf.org/html/rfc3253

Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Frank Karlitschek

On 22.03.2010, at 09:29, Cosmin Gorgovan wrote:

> On 03/22/2010 01:02 AM, Evgeniy Ivanov wrote:
>> Cosmin, I really enjoy discussion, but in my opinion it's beyond this
>> project. If you really willing to write user-space filesystem, I would
>> recommend you to focus on it as separate GSoC project (MINIX 3
>> organization is a good place for this).
>>
> OK, I've thought about it and I think you are right. I've also thought
> about other possible projects and I've came up with three smaller ones:
>
>   1. Implementing a interface to allow simple integration of new
>      database engines. Making it non-relational DB friendly would be
>      cool. Writing at least MySQL and sqlite modules.

Yes. But I plan to do this in the next few days. So I don't think this will be a good GSoC project.


>   2. Implementing multiple users and shared folders/files. Using WebDAV
>      ACL[1] seems the way to go for the communication part. The
>      database/database interface I mentioned earlier would require some
>      modifications.

This is also very easy and I plan to work on this soon. :-)


>   3. Implementing versioning. There's a WebDAV extension[2] for this
>      too. The database would require modifications again.

I think this could be a nice GSoC project. Integrating git or subversion as storage backend.
We should also do automatic cleanup of old revisions if storage is running out on the server.




> The last two items would require working in close collaboration with the
> desktop client developer(s) for agreeing upon the exact implementation
> details.

I would like to keep the client as simple as possible. So for a first step I
would do the versioning only on the server.
We could make the older version directly available on the client at a later stage. By doing this we only need webdav and don?t have to invent our own protocol. I?m not sure how good the WebDAV versioning support of the KDE kioslave is.

I?m a big fan of a lot of small steps. :-)


> So, what does everyone think?
>
> Thanks,
> Cosmin
>
>
> [1]: http://tools.ietf.org/html/rfc3744
> [2]: http://tools.ietf.org/html/rfc3253
> _______________________________________________
> Owncloud mailing list
> Owncloud at kde.org
> https://mail.kde.org/mailman/listinfo/owncloud


--
Frank Karlitschek
karlitschek at kde.org





Reply | Threaded
Open this post in threaded view
|

OwnClouds Future

Cosmin Gorgovan
On 03/23/2010 07:54 PM, Frank Karlitschek wrote:
>>    1. Implementing a interface to allow simple integration of new
>>       database engines. Making it non-relational DB friendly would be
>>       cool. Writing at least MySQL and sqlite modules.
>>      
> Yes. But I plan to do this in the next few days. So I don't think this will be a good GSoC project.
>    
I was thinking that might be wanted sooner.
>
>    
>>    2. Implementing multiple users and shared folders/files. Using WebDAV
>>       ACL[1] seems the way to go for the communication part. The
>>       database/database interface I mentioned earlier would require some
>>       modifications.
>>      
> This is also very easy and I plan to work on this soon. :-)
>    
At a first look it looks to me relatively simple but involving a fair
amount of work:

    * modifying the codebase to use auth from a DB instead of the config
      file
    * allow registration of multiple users
    * separate the webdav repositories per user
    * implementing read / write permission system
    * writing WebDAV ACL module in PHP(?) (yeah, without this it would
      be a lot less work)

>>    3. Implementing versioning. There's a WebDAV extension[2] for this
>>       too. The database would require modifications again.
>>      
> I think this could be a nice GSoC project. Integrating git or subversion as storage backend.
> We should also do automatic cleanup of old revisions if storage is running out on the server.
>    
Using git for storage sounds like a nice project. I found a GPL library
for Git written completely in PHP (http://github.com/patrikf/glip) but
I'm not sure if it implements everything that would be needed. Still,
considering it was used for the storage engine of a Wiki system, I'd say
it probably does or at least could be a good starting point.