This is an update on the progress of Metis as well as highlight some planned dates.
As it stands, Metis is currently a working prototype. It is a bit fragile and it doesn't take much to get it to keel over, but it works, and that is the most important part.
But without a doubt, there is still a bunch of work to do. We'll start highlighting those below:
As you can see, there is still quite a bit of work to do, and we'd like some ideal dates on when certain aspects of Metis 2.0 should fall into line.
2.0-alpha1: As much as I'd like to say "yes, go ahead and use the existing code for alpha testing", the reality it is a painful deployment and configuration process and you'd likely run into a bunch of known issues that we'd like to before recommending even testing Metis. We aim for alpha1 to be out on October 28th, 2015, and largely consist of error handling.
2.0-alpha2: After hitting the mark for alpha1, we'll continue working on error handling, but mainly focus on alpha2 cycle on the puppeteering and security model around that. The big picture for the CLI and puppeteering is that you'll be able to do per-Node/Node Group changes, which will modify the remote nodeList.json, or use it to modify a local nodeList.json that gets pushed to the server. You'll also be able to modify listening ports and get initially very basic status information. We aim for alpha2 to be out on November 18th, 2015.
2.0-beta1: This cycle will mainly be focused on cleanup of Go code in alpha1 and alpha2 cycles and continuation of the modernization of the Typescript code. We aim for beta1 to be out on December 2nd, 2015.
2.0-beta2: This cycle will be focused on final polishing of "all the things" and tackling build system, build instructions, and deployment instructions. We aim for beta2 to be out on December 16th, 2015.
2.0: The big 2.0 will arrive and this cycle will pull all of the work together into a cohesive package. We'll utilize the deployment instructions to create an official Dockerfile as well as a Docker container that will be built upon Fedora Server or Fedora Cloud (still to be decided which one) and the supervisord configuration (honestly the easiest part).
So after the big 2.0, what do we have planned? With certainty, I can say the following two items:
We will also evaluate support for Atomic images from Project Atomic and Nulecule spec. We will also evaluate support for Canonical's Ubuntu Snappy apps, also that will likely require supporting the use of external (as in not local to the container) configuration both in the CLI as well as Metis server, since Snappy apps are transactional and read-only.
This is all obviously in conjunction with bug fixes and whatever we come up with between now and 2.0! We hope all of this has been enlightening and we look forward to the fantastic cycles ahead.
This is an update on where we are with Metis. It will be broken apart across the different aspects of Metis, from the core Golang package, to the server and the client.
metis-pkg: We have been hard at work on metis-pkg. The initial write is complete however remains largely untested (we'll get into why in a sec). The idea behind the Golang package is other Go-based software, even our own Metis server, can utilize Metis. It also enables us to independently develop and version the core without affecting the rest of Metis.
Metis Server: Initially we planned on implementing "puppeteering" into Metis server. Basically, this would allow you to use the CLI or something else that sends an HTTP request along with the necessary security tokens, to remotely configure and control Metis. We've opted to put that on the backburner temporary to ensure that Metis server is tested as thoroughly as possible.
Metis' main HTTP server code, which handles API requests such as reading, writing, updating, and deleting files, has been implemented and needs testing. Which leads us to what is holding us back from doing that.
Metis Client: The Metis client code is in need of a cleanup and modernization, along with tweaks to reflect the move from a PHP to cleaner Go codebase. The plan currently is to do compatibility changes first, to ensure we can properly communicate with the server and do API requests. Once that is confirmed and bugs are ironed out in the server and core package, we'll work on cleanup and modernization of the client.
As it stands now, we should have a full-stack working prototype on October 21st and will likely have a blog post then to talk about it as well as detail a future roadmap. Until then, you are welcome to follow (and contribute) the metis-pkg and the future branch of Metis.
Hello everyone! Yesterday was a busy day for us, deploying not only Syiro 1.6.0-RC1, but also our Developer Center.
This blog post is about our plans for the Developer Infrastructure, our infrastructure, some background in what we use for specific functionality, with a ton of in-depth information. This blog is fairly technical, so beware!
The Developer Center is a home-grown platform built on top of Syiro and utilities markdown-it and highlight.js (with the Darkula theme) for parsing Markdown. Initially we had planned on using chjj/marked but due to the lack of activity, including resolution of bugs, we were not confident in its quality. markdown-it provides sufficient configuration options, is actively maintained, features numerous plugins in the event we wish to use them, and couldn't have been any easier to integrate highlight.js.
We opted to build a Go-based application to serve document metadata and the documents themselves. This ties into our nginx backend with a few configuration lines.
Our plans are to open source both the backend document server as well as the Developer Center sometime in November. We want to polish it beforehand and enables us to focus on Metis for our October cycle.
Some things we plan on doing:
Our infrastructure currently runs on top of Ubuntu 14.04 Server. We had chosen this prior to the release of Fedora Cloud and Server, and the rest of the VPSes (virtual private servers) we deploy use Fedora Server because it is vastly less chunky and uses systemd out-of-the box instead of Upstart.
Our website, including the Developer Center, is provided to end users via nginx as opposed to apache2. We currently have gzipped content enabled and for tying into our document server we use nginx's proxy_pass functionality to proxy /doc to the document server. The Document Server and nginx are controlled with supervisor.
supervisor provides us with the process controlling, launching, and grouping that we want. It is trivial for us to restart processes without having to touch PID1, whether it is Upstart or systemd. supervisor sits just above those core "userspace" applications and offers more flexibility than Upstart or systemd files do.
We reserve the option to do planned maintenance every Wednesday, except for this week which will be on Friday. Our plans specifically for this Friday are mainly configuration tweaks for supervisor and nginx. For example we currently don't have any cache controlling set and some assets aren't compressed as well as we'd like.
Eventually we'd like to switch the entire base over to Fedora Server and introduce a mirroring system that ties into gitolite, enables us to automate configuration and deployment of software, and applies the appropriate permissions.
To summarize everything: There is some fantastic open source software we're currently leveraging and we have plans to open source the document server and Developer Center in November. There is some infrastructure changes we want to implement over time, and we shared some of those with you, with insights on what our currently infrastructure is like as-is.
We hope this was insightful to you and maybe you discovered some open source technology to use yourself!
Hello everyone. We've been coding away and polishing the development center since the last blog post and we're are happy to announce that on September 21st, we will be deploying it.
Currently we are working on the landing page of the Development Center, with that and a few misc. documentation pages left on the documentation side. We'll be testing our mobile-scaled UX on a multitude of mobile devices to ensure the best range of device support over the weekend, landing whatever fixes we need to for launch.
Alongside the launch of the Developer Center, we'll be landing Syiro 1.6.0-RC1 (probably later in the day), which the Development Center will actually be based on. We have a lot of refinements in store in 1.6.0-RC1 and we look forward to the release announcement on our GitHub page.
Below is a teaser image of the Developer Center running locally, tied to our Go-based backend (which provides metadata and the documents). We'll have a more in-depth blog post-launch about everything we used to build this experience.
We will likely be able to deploy Developer Center without any sort of downtime / maintenance and will have further information via social networks around the time of launch.
On the hot seat of Syiro 1.5.3, which is the third and final bugfix and polish release of the 1.5 cycle, we have a lot in store for September and October.
There is good progress being made on the new wiki system. We are currently finishing the documentation fetching code and some tweaks to the backend server, as well as working towards making it more "generic", with some initialization options that should make it more flexible in usage. Our current aim is to get testing finished this week, with the debut on our site mid to late next week. We will be evaluating the process of open sourcing the development center and backend after that.
The plan for Metis in September is the initial prototype of the new server code and modernizing the client code. We recognized some design issues with the client code as well, thus those will addressed after the modernizing.
Once both the client-side Typescript modernizing and the new prototype Go backend are complete, we will plan our release schedule. We are "hopefully optimistic" for the 2.0 alpha, beta, rc 1/2, and stable releases in October. Likely it will be the fastest rapid release cycle we've ever done, and we look forward to getting it out there door.
And with that, we shall let the coding recommence!
We would like to announce that after the release of Syiro 1.5, we will be pausing Syiro development for approximately 4 weeks to work on our new developer experience, mainly a homebrewed wiki system that will enable us to be more flexible in how we present and structure the developer experience of getting started and using Syiro. We will be moving our documentation off our GitHub as we get closer to launch and will post more updates as we progress post-1.5.
This 4 week "hiatus" will also enable us to more broadly brainstorm radical and new things we want to do with Syiro in 1.6 and beyond. We have some non-finalized internal ideas and usecases we're going to tackle during the hiatus, so you can expect a separate blog (compared to the developer experience blogs) in the future as well.
Going forward, our plans for the 1.6 release cycle is to begin picking up development again on September 7th, rolling out RC1 on September 21st, RC2 on October 5th, and final on October 19th. Syiro 1.7+ release schedules will be changed accordingly.
We are happy to announce that we have moved our infrastructure over to DigitalOcean. This will enable us to be more elaborate in the services we will provide and enables more choices in the backends we intend on using.
Metis has evolved extensively in its 14 release history. It has endured a plethora of changes, a complete rewrite of both the server-side and client-side code, significant modularity (primarily in the client-side code), and use throughout a multitude of frontend apps and backend infrastructure.
But no matter how much innovation we have put into Metis, we have always been limited by the capabilities and performance of the language the backend was written in,
we've wanted to make Metis as modular as possible, the lackluster Class system of PHP has held us back. Where we've wanted native support for not just the Linux ecosystem, the additionally needs of
PHP have been a roadblock. We have desired to utilize concurrency to dramatically improve performance, we simply haven't been able to reliably do so without calling upon PHP libraries that we simply can't
expect everyone to have.
In order to introduce significant performance improvements, integrate with software such as Memcached, and be able to quickly innovate, we need to separate ourselves from PHP. Today we are
announcing that Metis 1.4.7, the current release, will be the last release of Metis in PHP. Starting on March 23rd, 2015, work will begin in a new
future branch in a language we have come to
Metis 2.0 will be written in Go. Go is a no-compromise, modern programming language that has been fundamental to a lot of incredible technology we know, like Canonical's Ubuntu Juju, or the container software called Docker. Go will provide us the high-performance we need through asynchronous function calls as well as Go's Channels model. Go has incredible support for other operating systems, the ability to compile down the Go code into an executable binary, has a fantastic set of core libraries, and has the community support we feel is needed to ensure we can integrate capabilities not seen before.
Metis 2.0 will have feature parity with the current PHP version, with obvious performance improvements along with changes to how querying the backend would work. We'd have Metis itself be a Go library so backend applications can utilize Metis without the need for the HTTP server.
Metis 2.1 would come roughly 6 weeks later and follow the same release schedule as Syiro. Metis 2.1 would feature a new file synchronization system and Windows Server support. We'd then move on to future versions of Metis and evaluate integrating support with software like Memcached.
We hope this provides some insight into the future of Metis. We're excited to see what this bright future will bring!