This document lists out things that I think Silver Lining should do, but does not yet do. It’s also kind of a notepad for design ideas. It should not surprise you if I forget to remove something from this document after implementing it.
It’s nice to present static content with far-future expiration dates. Do do this while still allowing this static content to be updated, it is conventional to use a timestamp of some sort in the URL itself. An example might be:
/static/20100127_1/media/style.css
Then if the application is updated this URL will change. You could use the modification date of the file itself, but a sweeping invalidation of all content is less error-prone and requires very little framework support.
I think Silver Lining should add a new location (like in the example) where content in the static/ directory is served with this caching, and an environmental variable $STATIC_URL which you use to prefix all your media files. Probably the current pattern will be preserved, as it can be handy when retrofitting an application. The datestamp_version used in the application directories is apppriately unique.
Then in the future there should also be support for uploading these static files to a CDN, at which point the variable will point to the CDN.
There are notes on this in ``multiserver design notes <multiserver.html>`_.
Backup and restore is implemented in a basic fashion. It would be very nice if backup files were portable. Then, for instance, you could download the backups to replicate live data on your site for your development. (PostGIS is a case where portable backups can be a bit tricky at times.)
It would be even more neat if backup files were somewhat version control friendly. E.g., well-ordered SQL dumps. This might make the backup process do double duty as a test fixture process, and a process to setup data in new instances.
Site-to-site backups should be supported (which would essentially clone the data from one server to another).
Log files should also be handled gracefully, tracked, and possibly some frontend (frontend in progress). Though I feel like some kind of management programming interface is better, and then a “deployed” app would use that interface. Actually, “interface” is probably saying too much; documented and conscious server layout that an application can be written to inspect.
There should be a way to get a list of all log files for a host/app. Maybe there should be a standard way to effect the Python standard logging module output, and then put it in a normal location (I think /var/log/silverlining/apps/NAME/*). There are some more notes down in Debugging.
Basically a way is needed to answer the question “do I need to rent bigger (or smaller) nodes?” Basic metrics like load, RAM, swap and requets per second should be monitored and graphed. Munin seems to be a popular choice for that. Probably a Nagios setup makes sense here. This post also has some additional ideas.
Also similar to update_fetch etc, there should be an optional defined “ping” URL for an application. This would be some location that could be safely fetched, wouldn’t be time-intensive, and would report general errors. An application could do self-testing here. Monitoring infrastructure would then be setup to ping this URL. Unlike update_fetch and other management URLs, this would be fetched in-process in the live server (management URLs get fetched with an analogous but non-mod_wsgi request). Probably there should be some indication of how often the pinging should happen (i.e., if the ping is cheap do it often, if it’s expensive be more reserved).
We should have a good quality rotating random value. (Like silversupport.secret.get_secret). Google has a library for this.
We should have a way to do security logging. E.g., log attempted attacks. There should be a whitelist, so that internal people can do bad things (e.g., when testing).
I’d like to integrate Pontoon into applications, so that Silver Lining can be a general way to setup applications for localization. This might primarily be a second deployed application that modifies files in the other application (somewhere in $CONFIG_FiLES).
Local development can for some people be quite hard. This is especially true of Windows users (for which silverlining doesn’t have any support, and no support is planned).
It would be nice to be able to run something like:
silver setup-develop-image . some-image-filename
and have it get a bare Ubuntu image (for VMWare, VirtualBox, or whatever) and run some adaptation of setup-node, and setup a file server to make editing files easy, and then upload the application into the machine (version control files and everything).
To make it more awesome, a built-in web app for managing the image; checking in files, restarting, tooling, etc. Maybe the image could be run headless then? This is more ambitious, but it could in some way be an introduction to good open source development practices.
This is an aside, but probably the generalized “DEST” idea of silver backup would be useful here and elsewhere: anywhere you write something (like an image file) allow remote locations. Of course silver backup is done on the remote server and uploaded from there, while this is done locally, so the advantage is smaller.
The core of this is probably two variations:
There should be a kind of “check functions are go” URL or command, similar to how update_fetch works. This would be fetched on update, but before the update was really made live (the internal request mechanism can be run on any instance, even one that is not live yet). If this returned a non-2xx result, then the update would be aborted.
Each update is a point when a persistence checkpoint should happen. This is most easily done with a local backup.
There should also be a “run all database updates” URL (also similar to update_fetch). This might fail, in which case the backup should be restored and the update aborted.
I think this should be run before test-before-activate, but if test-before-activate fails then the databases should also be reverted.
I’m fairly conflicted about what Silver Lining should do with respect to debugging. I am inclined to say that applications should be instrumented with debugging tools natively, e.g., using Django’s built-in tools or weberror, etc.
On the other hand, without this kind of wrapping you get something that really sucks – normal Apache error logs. Maybe at least environ['wsgi.errors'] should be fixed (along with stderr) to point to something better/reasonable. (And maybe warnings specifically should be handled, so only unique warnings are logged, and they are reset between updates.)
Setting up the logging module is somewhat similar, it could be done manually by applications. (Sometimes you want to turn up logging, in production temporarily to see what is going on in a live server: this is something not in the current workflow.)
Also there should be centralized configuration of things like, e.g., where error messages are sent. Then tools could be configured in a more transportable way (e.g., DEBUG_EMAIL = os.environ['SILVER_DEBUG_EMAIL']). This might fit into how configuration is generally handled.
Error reporting for non-requests (like a cron script) is harder. Most of these are done synchronously, so you can just show the user what is happening, but at least for cron and perhaps tasks this is not the case. Unifying environment setup might resolve this particular issue...
App Engine style cronjobs and tasks are definitely planned. (I also just found some code called PyPeriodic that I think I wrote a long time ago and forgot.)
Cron jobs are based on requests run periodically. Though I guess they could also be commands. One advantage of keeping to requests is to have some kind of monitoring for failed or rejected requests (though the internal request mechanism is not as robust currently as the external request system, which is managed by mod_wsgi).
Tasks are things to be done in the future, asynchronously with any request. These are queued, get retried in cases of failure (non-2xx response); maybe given multiple servers they could be run on different servers.
Cron is a higher priority.
I’m not even sure what this means, but I know it needs to be implemented in some way. I’m kind of holding off because I don’t want apps/servers to get configured a lot. But for reusable application packages something has to be implemented. And I suspect there are places where this is really sensible.
I will probably resist this until it is clear what it should mean, which means an actual circumstance where it is needed arises.
This is the obvious kind of non-portable configuration. There is some support for this already, but it is poorly exposed.
This is something that might be good to manage organization-wide. Which just makes the workflow slightly different.
This will only happen with a strong use case to motivate it; I can certainly imagine such use cases, but I need something more concrete than my imagination.
In that case, an application could receive configuration information for other applications. Communication would have to be through some service (e.g., a database) or HTTP. Accessing applications on other servers would probably not be supported (though potentially it could be), but given per-path applications it is reasonable to consider cases where this would be useful. Auth is an obvious case.
The Silver Lining secret is currently per-server (and will probably be per-group in the future), so some things like reading cookies could be done across applications already (given the appropriate code).
It would be nice to have a nice setup for getting test data setup. I think some degree of testing support would be nice, though simply installing and using a standard test runner seems mostly fine. A recipe for integration testing would be nice. A recipe for full-stack testing (e.g., with Windmill or Selenium) would be nice. These are probably more recipes than core Silver Lining things.
It’s possible that Silver Lining could setup a second set of “testing” services. So if you ran development off a database named myapp, maybe in “testing” mode it would use test_myapp (and the database would be cleared on every run).
I can imagine a through-the-browser development environment, based on Bespin. This could be cool. It could be really VM-friendly as well. Circumstances may determine if I really try this out.
To do this, Bespin would be installed and running, and configured to point at the application location. Version control can be managed through Bespin.
Additionally Hudson could be installed, enabling a kind of built-in test suite. I’m not sure how else Hudson could be enabled (except perhaps to list a test script in app.ini and go from there), but specifically for this case it seems like it would be nice. By including a test suite configuration it might generally be possible to do a very simple and easy Hudson setup.
Another interface would control Silver Lining itself, allowing you to configure production providers and deploy to them. It could be very simple.
Hudson is available as a third-party Debian package. Bespin would probably be deployed as a Silver Lining application itself. I imagine you might define a base domain (e.g., mydevel) and then you’d have:
One could imagine a load tester somewhere in here too. And maybe a rig for Selenium or other through-the-web tester?
With all these, heck, maybe I’d develop through a virtual machine even though I’m on Linux already.
Bespin would not preclude file sharing so that you could do editing (even version control) as though it was local. It would simply be another alternative.
Right now this system emphasizes development. It would be nice to also support ready-to-go applications. This might be simply a zip file created from an application, that can be easily uploaded. It might not be something at all fancy. But thinking it out would be nice.
Trac would be an ideal candidate for sometihng like this. It also has enough customization that I can imagine some interesting use cases; how do you select the plugins you want and the templates you want to override? If you aren’t “developing” the application then this isn’t as clear.
Maybe on application upload you could have two directories; one the “code” and another the “customizations”. The code would have a script to check and merge in the customizations before upload. One can imagine other useful things the code could do with the customizations (for instance, support plugin installation through knowledge of the app’s plugin index and techniques).
Maybe it would look like:
silver update someapp.zip --customize=mycustomizations/
That’s the easy way. Plugin installation?
silver manage-app someapp.zip --customize=mycustomizations/ \
--list-commands
silver manage-app someapp.zip --customize=mycustomizations/ \
install-plugin Akismet
It feels a bit crude. Maybe another command besides silver for pre-packaged apps? Or another set of subcommands.
It’s hard to figure out exactly what “version” of an application is installed somewhere. There’s no particular line, and no one thing is the only important version number.
It’d be nice to look around the files for VCS data and get a version snapshot for all pieces.
Another option is to have a deploy-application-registry, which would hold all the files that were deployed. This would be a parallel version control system unrelated to any developer version control, and would be used only for auditing.
I’ve done some applications that are mostly static pages on the frontend with Javascript. I like to use Google’s hosted versions of these libraries. But then offline I lose access to the app :(
It would be nice to have a local cache of these files in Silver Lining. Except I can’t figure out how you’d write a static HTML file that could select the local or remote file. Maybe one option is to use Google’s loader, but kind of mock it out for local use. But even that is tricky. It would be possible to do a small amount of rewriting of HTML pages, replacing those links (maybe literally) with local versions. During development static files are served fairly naively, so it wouldn’t be hard to do this substitution. Also minimized files could be replaced with unminimized files. Because the actual HTML would be production-ready there’s no compromise, only in development are they reverted.
Silver Lining could ship these libraries, or maybe there could be an app.ini setting like:
[development]
local_html_replacements =
http://ajax.googleapis.com/ajax/libs/jquery/1.4.0/jquery.min.js /_devel/jquery.js
Then you’d put that file in static/_devel/jquery.js. I can’t even get that on a single line... maybe these replacements should be put in a separate file, and maybe that should be a set of regular expressions. On the other hand, if this is really restricted to URLs and local files, you could “sync” these files automatically. But in this case, as an example, you actually want a different file during development (the non-minimized jquery.js).
Also while this is otherwise kind of unresolvable for static files, it would be nice if this also worked for dynamic files. There would then have to be some API. Maybe each of these resources would be “named”, in a file like:
[jquery]
production = http://ajax.googleapis.com/ajax/libs/jquery/1.4.0/jquery.min.js
devel = /_devel/jquery.js
And then an API like:
<script src="${tcsupport.href['jquery']}"></script>
Using __getitem__ would make this more Django-template friendly I believe (though a more traditional tcsupport.href('jquery') could also be supported).
This could also be the basis for the CDN setup, and it’s possible that minimization could happen using this. Maybe like:
[js]
jquery.json file = static/js/jquery.json.js
app.js file = static/js/app.js
all_js combine = jquery.json app.js
Then in production tcsupport.href('all_js') would return a URL pointing to the minimized and combined version of all those files, and tcsupport.href('jquery.json') and tcsupport.href('app.js') would both return None. In development the inverse would be true.
Using silver serve there should be an option to do minimization, so the minimized versions could be tested. The same ideas apply to CSS. Something like this might make sense for images, but no minimization would occur. You might not want to enumerate images, so you might do:
[images] images directory = static/images/
And then use ${tcsupport.href('images')}/folder.png
There should be a way (probably via ~/.silverlining.conf) to indicate how to start local services. Then, for example, you would not need to run PostgreSQL or Couch all the time, but could make it available as needed. Maybe it would look like:
[service.couchdb]
start = sudo /etc/init.d/couchdb start
stop = sudo /etc/init.d/couchdb stop
Of course starting and stopping all the time is annoying, but it wouldn’t have to happen if the reloader did the restart. Also perhaps Silver Lining could lazily stop the service by daemonizing a stop-script on exit. (Maybe only if there’s an additional lazy_stop setting.)
It would be good to auto-detect when setup-node has to be rerun. I imagine either simply using a hash of all files that are part of setup-node, or using the latest timestamp. The timestamp would change from computer to computer though, giving false positives.
I imagine if you run silver update that it would do something like:
$ silver update --node=foo
Warning: the node foo is not up to date; setup-node should be run
Continue anyway (y)es/(n)o/(u)pdate?
Yes (which would work with --yes) would just keep going, No aborts, Update would run setup-node immediately and then continue.
You can use mod_negotiation to serve static files without extensions.
Content Security Policy seems like you might want to enable it globally (not as part of the application). Specifically for static file headers.
Here’s what I want to do right away: