Yesterday at PhoneGap Day EU (sooooo sorry I'm missing it!), someone (I forget who) announced two new plugins for PhoneGap development - Push and ContentSync. Push is what you would expect - a way to deal with push messages easier. ContentSync is another beast altogether. The plugin makes it easier to update your application after it has been released. The API gives you a simple way to say, "Hey, I want to fetch this zip of crap and use it." It handles performing the network request to a zip, downloading it, providing various progress events, unzipping it, and then telling you where it stored stuff. All in all a kick ass plugin, but I had some difficultly understanding it so I worked on a few demos to wrap my mind around it. Before we get started though, let me clarify some things that were confusing to me. (And yes, I've filed some bug reports on where I got confused for possible documentation updates.)
- The first example shows this:
var sync = ContentSync.sync({ src: 'http://myserver/assets/movie-1', id: 'movie-1' });
What you may not realize is that URL you point to must be a zip file. So obviously a zip file need not end in .zip, but it wasn't clear at first that this was a requirement. - The plugin will unzip the file for you. Again, this is probably obvious, but it wasn't to me. The
id
value provided in the example above actually ends up being a subdirectory for where your assets will be stored. - The docs say that when the sync is complete, you will be given a path that is "guaranteed to be a compatible reference in the browser." What you're really given (at least in my testing in iOS) is a complete path to a directory. So if your zip had 2 files, a.jpg and b.jpg, to you could get the full path to a.jpg by appending it to the value. But this is not a 'browser compatible' reference imo. Rather you need to change it to a file URI if you wish to use it with the DOM. (To be clear, I could be wrong about this, but that's how it seemed to work for me.)
- By default, the plugin will always sync if you tell it too. You can pass an option to specify that it should only cache if a local copy doesn't exist, but for more complex logic like, "sync if the remote source is newer", then you need to build that logic yourself. That seems totally fair, just want to make it clear.
Ok, so how about an example? I decided to build a sample that would fetch a zip of kitten images. Here is where I made my first mistake. I took my folder in OSX, right clicked, and selected compress. This created a zip of one item (actually two, one was a system file) where the one item was the folder. That was not what I intended. What I should have done is select all the images, right clicked, and created a zip from that. I then put up the zip on my S3 bucket at https://static.raymondcamden.com/kittens.zip. For my first example, all I wanted to do was sync the zip and display them in the DOM. Here is the JavaScript code for this version:
//where to put our crap
var imageDiv;
//zip asset
var imageZip = "https://static.raymondcamden.com/kittens.zip";
document.addEventListener("deviceready", init, false);
function init() {
startSync();
}
function startSync() {
imageDiv = document.querySelector("#images");
var sync = ContentSync.sync({ src: imageZip, id: 'kittenZip' });
sync.on('progress', function(data) {
imageDiv.innerHTML = "<p>Syncing images: "+data.progress + "%</p>";
});
sync.on('complete', function(data) {
console.log(data.localPath);
var s = "<p>";
for(x=1;x<=7;x++) {
var imageUrl = "file://" + data.localPath + "/kitten"+x+".jpg";
s += "<img src='"+imageUrl+"'><br/>";
}
imageDiv.innerHTML = s;
});
sync.on('error', function(e) {
console.log('Error: ', e.message);
// e.message
});
sync.on('cancel', function() {
// triggered if event is cancelled
});
}
So for the most part, I assume this is self-explanatory. My zip file had seven images named kitten1.jpg to kitten7.jpg. Since I knew exactly what they were, all I needed to do was iterate and create img tags for each. This worked perfectly. I really don't need to share a screen shot of this. You already know it's seven pictures of cats. But you know me. I've got to share cat pictures.
Pretty darn easy, right? In case your curious about handling a zip of unknown images, you could use FileSystem APIs to iterate over the entries:
window.requestFileSystem(PERSISTENT, 1024 * 1024, function(fs) {
window.resolveLocalFileSystemURL("file://" + data.localPath, function(g) {
//ok so G is a directory ob
var dirReader = g.createReader();
dirReader.readEntries (function(results) {
console.log('readEntries');
console.dir(results);
});
}, function(e) {
console.log("bad");
console.dir(e);
})
});
Ok, so what if you only wanted to sync once? That is incredibly difficult unfortunately. You have to change
var sync = ContentSync.sync({ src: imageZip, id: 'kittenZip'});
to
var sync = ContentSync.sync({
src: imageZip,
id: 'kittenZip',
type:'local'
});
Yeah, that's it. Nice, eh? Now I'm not sure how often you'll have a sync strategy that simple, but it's great that the plugin makes it that simple. But what about a more real world example? Consider the code block below:
//where to put our crap
var $imageDiv;
//zip asset
var imageZip = "https://static.raymondcamden.com/kittens.zip";
document.addEventListener("deviceready", init, false);
function init() {
//determine the lastmod for the res
$imageDiv = $("#images");
$.ajax({
url:imageZip,
method:"HEAD"
}).done(function(res,text,jqXHR) {
var lastMod = jqXHR.getResponseHeader('Last-Modified');
console.log(lastMod);
if(!localStorage.kittenLastMod || localStorage.kittenLastMod != lastMod) {
console.log('need to sync')
startSync();
localStorage.kittenLastMod = lastMod;
} else {
console.log('NO need to sync');
displayImages();
}
});
}
function displayImages() {
var s = "<p>";
for(x=1;x<=7;x++) {
var imageUrl = "file://" + localStorage.kittenLocalPath + "/kitten"+x+".jpg";
s += "<img src='"+imageUrl+"'><br/>";
}
$imageDiv.html(s);
}
function startSync() {
var sync = ContentSync.sync({ src: imageZip, id: 'kittenZip' });
sync.on('progress', function(data) {
$imageDiv.html("<p>Syncing images: "+data.progress + "%</p>");
});
sync.on('complete', function(data) {
//store localPath
localStorage.kittenLocalPath = data.localPath;
displayImages();
});
sync.on('error', function(e) {
console.log('Error: ', e.message);
// e.message
});
sync.on('cancel', function() {
// triggered if event is cancelled
});
}
In this one I'm using localStorage to remember both the last modified value for the zip as well as I where I stored the assets. I perform a HEAD operation against the zip just to get the last modified value and if it is different from my last request (or doesn't exist), then I do a full sync. Once done we just run a simple function to iterate over the items in the local copy. Since I store the path I'll have access to it when sync operations can be skipped.
You can find all three examples over on my GitHub repo: https://github.com/cfjedimaster/Cordova-Examples/tree/master/contentsyncexamples. Also note that the plugin does even more than I touched on today so be sure to read the docs to see more.
Archived Comments
Very nice Raymond! Thanks for the tutorial. Do you know if it's possible to update html / JS files using this new plugin? That would be great, we only would have to resubmit the app to the stores only when changing major things...
In the docs they mention this specifically and in the demo app in the github repo, I see an example where after the sync they do a document.location.href to load it. I don't think that is a good idea though as you would need to wait for deviceready to load again, *however*, you could load in the new HTML via Ajax. I think a demo showing that type of thing would be useful - but I'll maybe wait for them (the project admins) to build that as I'm not sure I'd do it properly. :)
The Push plugin you linked to is quite old but it might have been updated recently. The readme still says "This plugin is a work in progress and it is not production ready." ContentSync looks great. Presumably we should be mindful of deleting old content from the filesystem so the storage demand doesn't grow excessively.
I'm in a rush but will fix the link tonight. I know a new one was mentioned.
Actually - it does show updates in the past two weeks.
Nice tutorial! I hope it's okay to post this url http://simonmacdonald.blogs... which contains slides of a demo Simon Mac Donald gave on the PhoneGap EU day this week.
Absolutely.
Thank you for this Blog, your Robust Mobile App Blog, and the Appcache with PhoneGap Blog!!! Those blogs have saved me time, agony, and possibly my job, lol. You rock dude!
For those using PhoneGap Build, here you go! https://build.phonegap.com/...
This will save many hours of Googling, and configuring! Also, this version of the plugin does not require the PhoneGap version in your config file to be 5.0 or higher; meaning, your javascript & config files do not need to be modified to comply with PhoneGap v5.0 requirements! You're welcome! "Keep On Programming, Programmers" - M. Stewart
This phonegap Build Plugin Not seems to be compatible with the 64bit. Az least thats what the error Message Shows After Building the App. And ideas?
It is a plugin, not a PhoneGap Build plugin. I'd report it on the Git repo.
Hey Raymond, thanks for your answer. Newbie mobile dev points to a phonegap Build Plugin. That's why i asked.
Ok - maybe I was being picky in my response. ;)
Great! Thank you. But I'm experiencing some trouble while trying to download a .epub file from my REST service. Before that I used GET request similar to this one: 'https://test-api.books.com/... and it worked fine. But now I get "Invalid request method" from my server. Will be very glad if you could help
Err, I'm confused. Are you using the plugin?
Thanks for this write up. What is a reasonable amount of content to be synced with this feature? I have subdirectories in my www folder that contain hundreds of short HTML files that need to be overwritten. Is that reasonable to attempt with this? I am a slow coder so I would hate to finally get it coded and find out its only good for small archives of kitteh pictures. :D
I think that is a difficult question to give a firm answer on - so forgive me if I waffle a bit. I'd begin by looking at a sample zip. How big is it? 2 megs? 5? Then determine how long it would take to fetch that data. There's a couple of ways you could test this. One would be to just do an XHR to it in a desktop web app but use Chrome's Network Throttling feature. Of course, that just gives you an estimate, but it could be useful to see if this is reasonable.
One thing about this particular solution is that it is has the benefit of one network request and the drawback of one network request. You get it all in one request, but the size is (possibly) huge versus smaller updates.
Give you are updating HTML files - I'd ask - are you always changing a *lot* of them at once? Do you really have to? So for example, imagine you have a common footer in the HTML doc that changed. Instead of including that in the HTML, include it in a template that could be loaded by your app. (So to display HTML Page A, it loads a.html + footer.html and perhaps caches footer.html in ram.)
Does this give you some ideas?
Thanks for the quick response. That does help me. The single, large network request is okay for our implementation. The zip is just over 100 MB and will continue to grow.
I'd reconsider then. Perhaps try to build an API that would let you get a delta (files changed since date X) and work with that.
I've sent a PR to improve the description of ContentSync to make it more obvious that it's for **zipped** content, not individual files.
Great - thank you!
Hello, I'm using this plugin for a project and works fine in the emulator but when I emulate in my device (iPad), I've got an error indicating invalid URL, I checked the whitelist (access origin = *) and it looks good so not sure what else it could be
Did you try editing the CSP value - that could be it.
Thx, I missed that
Sorry to bother you but I thought that I successfully fixed the issue but I'm still having the same issue, this is the CSP that I'm using <meta http-equiv="Content-Security-Policy" content="default-src * gap://*; style-src 'self' 'unsafe-inline'; script-src *; media-src *; img-src *; connect-src *; object-src *"/>
Yeah, that looks right. Not sure. You could reach out to the plugin creator and ask them. Outside of that I'd need to work on your code myself, and unfortunately that would need to be a paid engagement.
Hello Raymond is there any more update on this plugin and is it possible you can confirm this plugin can add new HTML and js and image from the zip file downloaded from the server in Cordova www folder
It would be best to check with the author of the plugin. I'm not doing much hybrid these days. PWAs rule now. :)