CryptPad funding status November 2018

I’ve now let a few months pass since the last August status. I’m sorry about that as I have been very busy with XWiki SAS activities as well as conferences including CryptPad talks.

I use this occasion to post the links to the different talks we had to present CryptPad and our goals:

We have also been busy preparing some other funding opportunities for CryptPad which we believe can help bring more development to the project. Rest assured we are hard at work to make it happen. By the way, if you are an independent developer in Europe and you would like to be funded to work on a CryptPad project, contact us, we could have an opportunity for you.

Also I wanted to announce that in addition to the NGI Award “Privacy and Trust Enhanding Technologies” received last month for CryptPad, XWiki SAS has now also received an award at the Paris Open Source Summit from the CNLL (Conseil National du Logiciel Libre). This award “Enterprise Award for an Open and Ethical Digital” (Prix de l’Entreprise pour un numérique ouvert et éthique), recognizes XWiki SAS’ involvement in sustainable free software development for the last 15 years. This of course includes our involvement in CryptPad and working hard to bring this free software solution to life and sustain its development. We are very proud of it and are committed to honor this award in the future.

Again we are grateful for our donators and subscribers as well as to all that relay our communications about the CryptPad crowdfunding campaign.

In the last three months we have received 1000 Euros from subscriptions and donations. this number has been growing in particular since we have launched yearly payments for subscriptions.

We have now exceeded 4000 Euros in funding for CryptPad since the beginning, including 1000 Euros of donations. This funding comes from 65 different sources from 19 different countries. 26 donators are coming through Open Collective.

cryptpad funding details cryptpad funding by type cryptpad funding by country

While this cannot yet fund our two developers we are happy that the funding is progressing. We hope to continue to grow subscriptions and donators and have between 6k and 10k Euros by the end of the OpenPAAS NG project which currently funds us. Progressively we are getting close to secure funding for the project.

Now we need more help ! First we need to be able to sustain our team, but also we need to be able to expand, especially if we want to have Open Source software in the Zero-Knowledge space. Proprietary closed source software providing some form of Zero-Knowledge will probably start growing with extensive funding. If we want independent free-software alternatives for this type of software, then we need to group our effort and fund open source solutions like CryptPad. We have ambitious objectives for CryptPad and we cannot achieve these with only two developers.

Since the launch of the campaign we have also published a roadmap for CryptPad of what we would like to achieve with the funding. It’s also available on the OpenCollective web site. Come check it out and see our ambitious objectives for this project. We are already making progress on this roadmap.

Aaron and Yann, our CryptPad developers, have released the shared folders feature, one of the most demanded feature. We have also worked on improving the reliability of the synchronization engine (chainpad).

To finish, I’d like to give progress on CryptPad’s usage. We are now reaching close to 200 instances of CryptPad running around the world, and the official cryptpad.fr instance is growing regularly. We have seen a surge in the last weeks, maybe related to the conferences, and we have now more than 1000 drive users, and 6000 pad visitors per week. The activity has almost doubled in 4 month.

cryptpad drive stats cryptpad pad stats

Try CryptPad, love it, take care of it, and even better come help !

Ludovic Dubost & the CryptPad Team

CryptPad receives NGI Startup Award

Europe recognizes Privacy, and it’s starting with CryptPad

Europe’s Next Generation Internet initiative (NGI.eu) awarded CryptPad the Next Generation Internet’s Privacy and trust-enhanced technologies startup award. The NGI Startup Awards recognize Europe’s most disruptive entrepreneurs who are advancing revolutionary products, solutions and services destined to have a major impact on the internet of the future.

Of course XWiki SAS and the CryptPad team are very proud of this award for a project that was started 4 years ago by Caleb James DeLisle as part of a French research project funding realtime editing in XWiki. The first commit and so it begins was premonitory of a long and important project.

But most importantly we are very happy that privacy enhancing technologies are getting recognized. We strongly believe that users of internet technologies should be able to control their privacy and the security of their data, and unfortunately this is not the case today. The dominating business model of the Internet is advertisement based on exploiting user data (“Your data is their data”). Cloud providers have full access to most of users’ and companies’ private data, which is being exposed to many risks.

Our promise is that our software cannot spy on its users, and that your data is safe.

We are also very happy that our users and supporters are recognized. Since we launched our online service CryptPad.fr as part of the OpenPaaS-NG research project, funded by BPI France, CryptPad’s usage and support has continously grown. With more than 10000 users weekly and also the support from paying users and donations through our Open Collective, we are all participating in helping restore our privacy.

It is difficult for all of us to give up powerful Internet services and software which bring us great value, but at the same time we do not like to see how our data is being used for advertisement, political means or malicious hacking. Today this NGI Award is showing that it is possible to take back our privacy, while getting powerful and easy to use services. We built CryptPad to show how far a team can go to empower users and increase their expectation of privacy from online services. While it was previously accepted that collaborative editing meant sacrificing confidentiality, we’ve not only proven that private editing is possible, but we’ve made our entire platform open source to ensure that this technology remains available.

Want to be a part of it?

  • Use CryptPad and other Zero Knowledge services every day, tell us what you like and what we can do better.
  • Talk to your friends and colleagues about Zero Knowledge, show them CryptPad and explain that this is what the cloud can be.
  • Candidate to XWiki SAS to join our team.

Show your support

  • Buy an upgraded account from Cryptpad.fr, run by the CryptPad development team, or contribute to our Open Collective
  • If you install the Open Source code of CryptPad on your own servers, consider buying a support contract.
  • If you’re a web developer, think about Zero Knowledge for your next web app.

About CryptPad and XWiki SAS

CryptPad is an open-source, web-based suite of collaborative editors which employs client-side cryptography to ensure that the server is not able to access the contents of users’ documents. It offers a variety of editors and other multi-user applications: rich text, code editing with syntax highlighting and markdown preview, presentations, polls for scheduling, kanbans for project management, and whiteboards for collaborative illustration.

CryptPad is being actively developed by XWiki SAS and is currently funded by BPI France as part of the OpenPaaS NG research project. For the last 14 years, XWiki SAS has built Open Source Collaboration Software and provided professional services allowing organizations to better organize their information.

About the NGI Initiative and awards

NGI is Europe’s new approach to creating a more human-centric internet. It invites citizens and communities striving for values like openness, inclusivity, transparency, privacy, cooperation, and data protection to provide input, and thus to help to guide the European Next Generation Internet funding agenda. NGI is a European Commission initiative which is being implemented by project partners throughout Europe.

The overall mission of the Next Generation Internet initiative is to re-imagine and re-engineer the Internet for the third millennium and beyond. We envision the information age will be an era that brings out the best in all of us. We want to enable human potential, mobility and creativity at the largest possible scale – while dealing responsibly with our natural resources. In order to preserve and expand the European way of life, we shape a value-centric, human and inclusive Internet for all.

CryptPad funding status August 2018

We have now passed the summer since we have started the OpenCollective for CryptPad and started our campaign for funding CryptPad and ensuring its future in 2019, so I wanted to give an update on this funding initiative.

First we are highly grateful to all that have responded positively and also that have relayed our campaign. We do feel stronger with the help of our community and it motivates us more.

We have now received a few more contributions during this month of August, and in particular a 100 Euro donation from Matthieu Herrb. In July we had received a commitment from the C3Wien Community in Austria, to contribute 360 Euros / year as well as some of their members to whom our communication was relayed. We have also received a few donations for more than 50 Euros coming from the USA and Germany (thanks Michael Goelzer, KimoNine, shibayashi) and also monthly donations (thanks Daniele Gagliardi and an anonymous backer !) and one time donations reaching now more than 10 donators through OpenCollective.

We also had new subscribers to the CryptPad subscriptions and in total now we can count on 40 participants to CryptPad funding for a total now of 320 Euros for the month of August and 2250 Euros since subscriptions started last year. This funding originates from 11 different countries around the world.

funding by country

XWiki SAS is also matching the donations we received since the beginning of the campaign and we are now reaching a budget of 2614 Euros on OpenCollective.

This amount is of course still low and not enough to guarantee funding next year for CryptPad, but it’s a start without much communication about the funding of CryptPad and about the long term objectives of this project. We still have 7 months to grow this funding to contribute to the development of CryptPad.

Now we need more help ! First we need to be able to sustain our team, but also we need to be able to expand, especially if we want to have Open Source software in the Zero-Knowledge space. Proprietary closed source software providing some way of Zero-Knowledge will probably start growing with extensive funding. If we want independent free-software alternatives for this type of software, then we need to group our effort and fund open source solutions like CryptPad.

Since the launch of the campaign we have also published a roadmap for CryptPad of what we would like to achieve with the funding. It’s also available on the OpenCollective web site. Come check it out and see our ambitious objectives for this project. Aaron and Yann, our CryptPad developers, are already hard at work and the shared folders feature, one of the most demanded features, is already advancing very fast. During the summer there are has been 3 new releases of CryptPad

Try CryptPad, love it, take care of it, and even better come help !

Ludovic Dubost & the CryptPad Team

CryptPad funding status after one month

one month funding status

It’s been a month now that we have started the OpenCollective for CryptPad and started our campaign for funding CryptPad and insuring it’s future in 2019, so I wanted to give a status on this funding initiative.

First we are highly grateful to all that have responded positively and also that have relayed our campaign. We do feel stronger with the help of our community and it motivates us more.

In particular we have received a commitment from the C3Wien Community in Austria, to contribute 360 Euros / year as well as some of their members to whom our communication was relayed. We have also received a few donations for more than 50 Euros coming from the USA and Austria (thanks Michael Goelzer, KimoNine, shibayashi) and also monthly donations (thanks Daniele Gagliardi and an anonymous backer !) and one time donations reaching now more than 10 donators through OpenCollective.

We also had new subscribers to the CryptPad subscriptions and in total now we can count on 30 participants to CryptPad funding for a total now of 750 Euros for the month of July and 2000 Euros since subscriptions started last year. This funding originates from 11 different countries around the world and actually not yet from France our home country who has funded the beginning of CryptPad with the OpenPAAS project.

funding by country

This amount is of course still low and not enough to guarantee funding next year for CryptPad, but it’s a start without much communication about the funding of CryptPad and about the long term objectives of this project.

Now we need more help ! First we need to be able to sustain our team, but also we need to be able to expand, especially if we want to have Open Source software in the Zero-Knowledge space. Proprietary closed source software providing some way of Zero-Knowledge will probably start growing with extensive funding. If we want independent free-software alternatives for this type of software, then we need to group our effort and fund open source solutions like CryptPad.

Since the launch of the campaign we have also published a roadmap for CryptPad of what we would like to achieve with the funding. It’s also available on the OpenCollective web site. Come check it out and see our ambitious objectives for this project. Aaron and Yann, our CryptPad developers, are already hard at work and the shared folders feature, one of the most demanded features, is already advancing very fast.

We want also to take the opportunity to thank Caleb James De Lisle, the creator of CryptPad, who has left XWiki SAS at the end of July to pursue his own projects (CJDNS and others). Caleb has done incredible work first creating CryptPad and also on XWiki SAS managing our research projects for the last few years. CryptPad’s development is in good hands with Aaron and Yann who have been the main commiters of the project in the last year. In the last few month he has helped setup the Open Collective and the roadmap to help fund CryptPad in the future. I will now relay him on this aspect and we are a looking for a new team member to join our research funding initiative and lead CryptPad on the non-technical aspects. I’m sure Caleb won’t fully leave us and at least be a regular adviser to the project.

Try CryptPad, love it, take care of it, and even better come help !

Ludovic Dubost & the CryptPad Team

Making CryptPad CSS 3 times faster, by loading it twice

In CryptPad development, we have always tried to push the limits of the technology. As you might know, we don’t minify any of our javascript code and we have no build system, yet CryptPad is still faster than many similar projects which do. In recent profiling, we determined that the biggest cause of slow loading was compilng of less stylesheets.

All of the styles for CryptPad are written in Less CSS templating language and because we don’t have a build system, when you load a page on CryptPad, it downloads the less compiler and runs it in your browser. When the less has been compiled the first time, it is cached in the browser’s localStorage so that it doesn’t need to be compiled again (until next release). Unfortunately, the way we structured CryptPad Less code led to this taking a long time.

CSS by its nature is very much like object inheritence, a design pattern popular in the 1990s which has since been discredited. In a bid to keep our styles under control, we decided to make heavy use of Less mixins. The idea was that we didn’t want our CSS code to “speak until it was spoken to”.

Something like the following would be very problematic if it were dumped on the global scope, but it is never output until it is called:

1
2
3
4
5
.example_header() {
a {
color: red;
}
}

So then the code which uses it can invoke it only in the exact place where it ought to be used.

1
2
3
4
5
6
7
8
@include "example.less";
.cp-app-pad {
.cp-padheader {
.cp-padheader-left {
.example_header();
}
}
}

If you’re a CSS purist, you’re probably pulling your hair out now, because the right way is to use html classes. The thing is, we do, but because CryptPad is made up of many different pieces of open source software, we cannot control all of the HTML and sometimes things aren’t so simple. Using mixins gave us an extra layer of safety and allowed us to write CSS feeling quite confident that it would not end up changing things where it wasn’t supposed to.

Cascading works great, until you include one CSS file that was written by this guy.

Parameterized mixins

An excellent feature of Less is parameterized mixins, that is, templates with arguments. One of our biggest templates is called .toolbar_main() and this builds the toolbar at the top of CryptPad as well as the user-list on the lefthand side. In order to make the customizable colors with the same HTML structure, we opted to use a parameterized mixin like this:

1
2
3
4
5
6
7
8
9
.toolbar_main (
@color: @colortheme_default-color, // Color of the text for the toolbar
@bg-color: @colortheme_default-bg, // color of the toolbar background
@warn-color: @colortheme_default-warn, // color of the warning text in the toolbar
@barWidth: 600px // width of the toolbar
) {
/// a lot of code here, using colors based on the parameters,
/// but lightened or darkened using less functions.
};

Then each app would call .toolbar_main() with its own color parameters, and get a nice toolbar, customized to that app’s color theme. Again, this is not the only way to do it, but having the ability to generate CSS with highly specific color rules proved to be extremely useful for overriding leaked styles coming from the software that we integrate.

State explosion

What we didn’t think about at that time was the effect that the proliferation of CryptPad Apps would have on the amount of CSS being generated. We started with just a few styles and just a few apps to apply those styles on.

But as we added more and more CryptPad apps, the same CSS was being generated and applied over and over…

Then as our styles become more complex, the CSS which was being copy/pasted by less compiler became bigger and bigger

The total of all our less code in the entire project was only 235k, and it was compiling to over 1.3 megabytes of CSS. We cache the compiled CSS by placing it in localStorage, but still, every time a new version of CryptPad was released, the CSS needed to be recompiled and this was dominating the loading time.

Building a Linker for CSS

If you have experience with C/C++, you might recognize this problem. It is as if there was no linker and the only way to reuse code was to use preprocessor #include over and over again.

Since by this point, we had a significant amount of Less which was designed this way, rewriting it was not an option, so we started looking for ways to link it rather than copy/pasting it over and over again. Fortunately most of the bigger mixins only applied rules to specific classes, so moving them up to the root level would not cause trouble, though to be safe, we wanted to only load the styles that were necessary.

If we would indicate to the javascript code which loaded the Less that a particular Less file was needed, it could be compiled, cached and included separately, and thus it could be reused across apps. In order to keep the Less API as close to the same as possible, we decided to put that indication inside of the .<filename>_main() mixin.

So .dropdown_main() went from this:

1
2
3
.dropdown_main () {
// all the code here
}

To this:

1
2
3
4
5
6
.dropdown_main () {
--LessLoader_require: LessLoader_currentFile();
};
& {
// all the code here
}

Two important things to note: firstly LessLoader_currentFile() is a function which we created (as the name implies, it’s defined in LessLoader.js), it simply expands to the current function name. Secondly, when a less file is included with the reference flag (e.g. @include (reference) "./dropdown.less";) the content is not output but the mixins are made available, so moving the code down to the bottom of dropdown.less would cause it not to end up in the compilation of app-pad.less.

The resulting CSS from this contains something like this:

1
--LessLoader_require: "/customize/src/less/include/dropdown.less";

LessLoader would then simply scan for --LessLoader_require: and trigger loading of those files, which are still parsed by Less, but are the same for every CryptPad app.

Parameters with CSS variables

In this example, I intentionally left out the parameterized mixins. Solving this was slightly more complicated and in order to do it, we made use of a reasonably new feature in web browsers: CSS variables.

Today, one can write in CSS the following:

1
2
3
4
5
6
7
8
:root {
--color-should-be: brown;
}

// potentially much later...
.element {
color: var(--color-should-be);
}

and the element text will be brown. Discovering this was a breakthrough because it meant that the arguments could be turned into variables in app-pad.less and then made use of in toolbar.less. However, there are limitations to what you can do with CSS variables. For example, this doesn’t work:

1
2
3
4
5
6
7
8
9
10
:root {
--hack-boolean: 1;
}

// later on...

@media screen and (max-width: calc(var(--hack-boolean) * 100000000)) {
// HAHAHA I MADE AN IF STATEMENT
:root { --lets-define-another-variable: "lol"; }
}
If this worked, I’d probably be using it


Scoped CSS variables

What does work, however, is specifying different values of the same variable at different scopes, so this does work:

1
2
3
4
5
6
7
8
.my-button { --button-color: red; }
.my-popup-window .my-button { --button-color: blue; }

// later on...

.my-button {
background-color: var(--button-color);
}

But possible uses/abuses of this feature were not investigated.

Making it work

Following the general principle of keeping the variable definition close to the usage, we put the variable definitions inside of the .<filename>_main() mixin and the usages below in the same file. In order to avoid namespace collisons, we prefixed all variables with the name of the file. A simplified version of avatar.less looks like this:

1
2
3
4
5
6
7
8
9
.avatar_main(@width) {
--LessLoader_require: LessLoader_currentFile();
--avatar-width: @width;
}
& {
&.cp-avatar {
...
.cp-avatar-default, media-tag {
width: var(--avatar-width);

In some cases, we needed to introduce additional variables because of the use of Less functions such as lighten() and darken(), which obviously cannot work on CSS variables. So we used the following pattern in many places:

1
2
3
4
5
6
7
8
9
10
.help_main (@color, @bg-color) {
--LessLoader_require: LessLoader_currentFile();
@help-bg-color-l15: lighten(@bg-color, 15%);
@help-text-color: contrast(@help-bg-color-l15, #fff, #000); //@color;
@help-link-color: contrast(@help-bg-color-l15, lighten(spin(@bg-color, 180), 10%), darken(spin(@bg-color, 180), 10%));

--help-bg-color-l15: @help-bg-color-l15;
--help-text-color: @help-text-color;
--help-link-color: @help-link-color;
};

Results

After carefully planning and studying solutions, we managed a fairly non-invasive refactoring of the styles which took Less compile time down from almost 3 seconds to around 900ms. For simplistic pages like the front page, the number dropped to around 200ms.

The key result is that a person who has never seen CryptPad before will see the main page right away instead of waiting 3 or more seconds to compile all the less for the entire project.

But wait, what about Internet Explorer ?

This question is the bane of many web developers’ existance. In this case, the problem is that Internet Explorer has no CSS variables. Last week we changed CryptPad so that when you use it, your browser will let us know if it doesn’t support CSS variables. This is done using the feedback mechanism which is an opt-out collection of information such as how often particular features are used and whether certain things are supported by the browsers of people using CryptPad.

What we found is that in the past week, we saw about 50 unique users who are running browsers which don’t support CSS variables. With our approximately 4500 unique users per week, this is a little over 1%.

Making an acceptable fallback

The One Percent jokes asside, it’s hard to justify making CryptPad 300% slower for everyone who tries it for the first time, just to satisfy about 1% of the userbase. But at the same time, it’s sad to drop support for a browser which at the current moment does work with CryptPad.

The solution we devised was to specify default values and then override them.

For example:

1
2
3
4
5
6
7
8
.cp-markdown-toolbar {
...
button {
// IE sees this (variable compiled by less)
color: @toolbar-color;

// everyone else sees this
color: var(--toolbar-color);

The only question remaining was how to specify the defaults in a way that was simple for us when we worked on the less files. Most of our .<filemane>_main() parameterized mixins already had default values in case they were called without parameters, so we already knew what sane defaults would be. What we decided to do was create a new mixin called .<filename>_vars(), which would assign a set of Less variables based on the arguments.

The final result looked like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
.help_vars (
@color: @colortheme_default-color,
@bg-color: @colortheme_default-bg
) {
@help-bg-color-l15: lighten(@bg-color, 15%);
@help-text-color: contrast(@help-bg-color-l15, #fff, #000); //@color;
@help-link-color: contrast(@help-bg-color-l15, lighten(spin(@bg-color, 180), 10%), darken(spin(@bg-color, 180), 10%));
}
.help_main (@color, @bg-color) {
--LessLoader_require: LessLoader_currentFile();
.help_vars(@color, @bg-color);
--help-bg-color-l15: @help-bg-color-l15;
--help-text-color: @help-text-color;
--help-link-color: @help-link-color;
};
& {
.help_vars();
.cp-help-container {

position: relative;
background-color: @help-bg-color-l15;
background-color: var(--help-bg-color-l15);

Going through it step-by-step, the .help_vars() mixin takes parameters but it defines default values, then it creates some Less variables which are accessed after it is used. .help_main() calls .help_vars() and passes it arguments, then takes it’s results and assigns them to CSS variables. Then the main block of the Less file also calls .help_vars() but without any arguments, so the defaults are used. Then in the main block, each usage of a CSS variable also has the usage of the less variable. The less variable provides the default value, the one which IE will see, and then the CSS variable provides the specified value, the one which will be different per CryptPad application.

There you have it

Every Less file now gets loaded twice, first it is loaded by the @include (reference) call, where it is parsed in order to expose the .<filename>_main() mixin, then it is loaded a second time as an independent file. Importantly, however, once it is compiled it is stored into localStorage and the toolbar CSS which is compiled when you go to the Rich Text app is the exact same CSS which will be used when you go to the Code/Markdown app. This feature is available in the staging branch of the CryptPad project and will be on CryptPad.fr in the next release.

We could make it even faster, by separating the main content of the Less files from the .<filename>_vars() and .<filename>_main() mixins, essentially making header files, in C/C++ parlance. And if we find in future profiling that less compiling remains a signficant performance penalty, there’s a good chance that we will.