Infrastructure

keeping prebuilt LO around is the tricky part with this proposal, for Hackfest, LO is built once when creating/setting up the VM and then left for the users.

For a constantly available VM, this of course won't work, and just having it build like tinderboxes do will conflict with users trying to use it.

So suggestions on how to solve this welcome. One idea is to build daily/regularly using a dedicated user, and copying/linking files to the user's home when the user logs in, maybe using a filesystem with snapshots that can be mounted separately.

Other problem to solve is who gets/grants access. How to reduce the risk of two users logging in as user1?
And how to clean up?

So bottom line: needs more refining of the proposal to get a working 24/7 solution.. A one-off is easier, although still tricky to get right in terms of user management/assignment. Just providing password (eek, password based login) or a ssh key on a website is not something I'd like to do for a permanent service (or otherwise put: I couldn't sleep well, always having to think whether the users might do stupid stuff with the VM/abuse it/making it part of a botnet...

So while it reads ok to have it "restricted as possible" - this is not an area where I have much experience with. Either you have user account and default permissions (user/group), or you don't have access is all I'm used to. Quota maybe in addition, but nothing that really comes close to locking down/preventing abuse.

Network abuse could be somewhat limited by only allowing creating connections to gerrit and dev-www, and only accepting incoming on ssh - but still not something that is bullet proof...

So Q1 for the thing likely is too ambitious.

Christian Lohmaier wrote:

keeping prebuilt LO around is the tricky part with this proposal, for Hackfest, LO is built once when creating/setting up the VM and then left for the users.
[...]
So suggestions on how to solve this welcome.

I think a prebuild LO once per week should be enough for a start. And I wouldnt force users to bump along: Let them stay on the VM checkout you started from and manually update as they go (yes, that suggests rebuilding themselves).

Alternatively they can freshly start from the new virgin VM image of the week and pull back their work from gerrit.

The risky part of the onboarding is up to the first patch is pushed, so having prebuild product there is essential.

Once the first patch is pushed, the contributor is already invested and having them do rebases/rebuilds isnt too bad: They'll need to at some point anyway.

Other problem to solve is who gets/grants access. How to reduce the risk of two users logging in as user1?

Someone at TDF needs to be the keykeeper. Dev mentor is the natural candidate.

And how to clean up?

I would suggest to not "clean up" these images, but run them as long as they work for a contributor. When they dont, throw away and start from a new virgin prebuild image of the week.

@JanIv: Can you scout out the pricing at CloudShare? E.g. for our use-case what are the expensive parts (updating an image, running an image, having an image laying around idle etc.)

Christian Lohmaier wrote:

So while it reads ok to have it "restricted as possible" - this is not an area where I have much experience with. [...]

But CloudShare should have, as what we are aiming for is pretty much their advertised scenario. I would be surprised if they hadnt considered this and have workable solutions for this (e.g. monitoring network volume and warn if it burns through some limits). Please contact with CloudShare (and other providers) and ask them for their thoughts on this.

Someone at TDF needs to be the keykeeper. Dev mentor is the natural candidate.

If a keykeeper is wanted I would of course happily do that as long as I am dev mentor

the getinvolved page would simply ask people to mail mentoring@ to get the details.

I will take a look at cloudshare as requested

Bjoern Michaelsen wrote:

@JanIv: Can you scout out the pricing at CloudShare? E.g. for our use-case what are the expensive parts (updating an image, running an image, having an image laying around idle etc.)

The team option seems to have what we want, 166USD/month and 66USD/month for extra users. Especilly the role based accounts would be good.
https://www.cloudshare.com/pricing

Just leaving this here:
https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/

Setting up servers to use with manually triggered cleanup can be done in our infra, assuming people are OK with using ssh-tunnel to connect to windows RDP (remmina client can do this quite nicely)

So if someone manages the access to the hosts and can signal: everyone is done with their stuff, OK to prune and prepare next round, then no problem with setting it up.

(the cleanup can be accelerated by having a "shadow" build that can just be copied over - but the problem on when to determine that the current working tree is no longer used/needed will require some "manager" to oversee, at least at the beginning/unless someone comes up with a more concrete to solve this programmatically :-))

re microsoft VMs: would not solve the "start immediately" problem, with enough bandwidth the user might be able to skip a few hours, but that would rely on having identical paths, etc. And installing the development toolchain would still take quite some time.

So, what are the next steps and do you need anything like budget
approval before? :-)

Last week's status:

-> Windows part is basically ready, we just might need some more VM disk space
      -> 5-7 concurrent users is possible, but rdesktop limitation
      -> AI: Cloph to look into the Cloud-something service Björn mentioned
      -> user management to be handled by Cloph and Xisco
      -> build tree update still needs automating
   -> Linux is easier, but not done yet
   => deadline for both is May, so ~one month to go

Quoting the last team call:

 * PENDING: hack fest vm to run 24/7 and be available to new contributors (#2136)
   -> Linux side is ready, Windows side needs VM cloning, will be done this week 

With the deadline approaching, I wanted to follow-up on this.

Cloph, has everything been setup properly?
If not yet, what's the ETA?

Adding Mike to this ticket, as I'd like him to come up with some blogposting/video/howto on that thingie. IMHO Mike has no experience yet with the build environment, which makes him the ideal candidate to get credentials and have a look to see if things work like a charm as planned or if there's a need to improve stuff. ;-)

Adding Sophie to this ticket, as she will lead the next team call and this should be talked through, given the deadline is approaching.

Cloph, in case you're not finished yet, please prioritize this. Thanks!

Just talked about this in the team call - Cloph is going to send me some instructions, which I can then test and identify any gaps. In addition, I could try making a video to show how it works.

Sounds great!

The Document Foundation Redmine wrote on 2017-05-02 at 15:44:

Just talked about this in the team call - Cloph is going to send me some
instructions, which I can then test and identify any gaps. In addition,
I could try making a video to show how it works.