Hi guys,

You should know I do have a fork of Arduino IDE (for another CPU), and I kinda do most stuff focused on my system, and I tend to neglect Arduino often, also
because Arduino development is a bit strict, thus not allowing many changes from version to next version.

Anyway, I'm facing a problem, and I do believe you are also facing a problem:

There's no way do do "incremental" upgrades on Arduino system.

Although release cycle in Arduino is large, it does not happen with my system, I've to constantly push changes to users, for the system, albeit stable,
sometimes requires changing at least the core libraries for size and performance sake.

Right now, someone wishing to upgrade his Arduino installation (let's say, from "1.0" to upcoming "1.0.1") has do download a large zip file, remove the older
installation, and extract everything again. This presents a problem when you want to push small changes - being required to download a 100Mb zip file just
because there's one line changed on a ".h" is, well, stupid.

So my idea is to build or integrate an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version (using
internet) to a more recent one (you can choose your next version), while eventually keeping your own "patched" files if you happened to mess with the IDE. Only
relevant files would be transferred.

This is something that I will implement, and something I want to give back to Arduino community. I still have not figured out all requirements (server and
client), but should have done so in a week.

How open are you to include this feature in some next release ?

Best,
Álvaro Lopes
http://www.alvie.com/zpuino/

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

There are a number of frameworks for this already; you might look at the (oft-maligned) WebStart, or something like http://www.interdirected.com/index.php?option=com_content&task=view&id=34&Itemid=61

= Mike

On May 11, 2012, at 11:34 AM, �lvaro Lopes wrote:

> Hi guys,
>
> You should know I do have a fork of Arduino IDE (for another CPU), and I kinda do most stuff focused on my system, and I tend to neglect Arduino often, also
> because Arduino development is a bit strict, thus not allowing many changes from version to next version.
>
> Anyway, I'm facing a problem, and I do believe you are also facing a problem:
>
> There's no way do do "incremental" upgrades on Arduino system.
>
> Although release cycle in Arduino is large, it does not happen with my system, I've to constantly push changes to users, for the system, albeit stable,
> sometimes requires changing at least the core libraries for size and performance sake.
>
> Right now, someone wishing to upgrade his Arduino installation (let's say, from "1.0" to upcoming "1.0.1") has do download a large zip file, remove the older
> installation, and extract everything again. This presents a problem when you want to push small changes - being required to download a 100Mb zip file just
> because there's one line changed on a ".h" is, well, stupid.
>
> So my idea is to build or integrate an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version (using
> internet) to a more recent one (you can choose your next version), while eventually keeping your own "patched" files if you happened to mess with the IDE. Only
> relevant files would be transferred.
>
> This is something that I will implement, and something I want to give back to Arduino community. I still have not figured out all requirements (server and
> client), but should have done so in a week.
>
> How open are you to include this feature in some next release ?
>
> Best,
> �lvaro Lopes
> http://www.alvie.com/zpuino/
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

Michael Smith wrote:
> There are a number of frameworks for this already; you might look at the (oft-maligned) WebStart, or something like http://www.interdirected.com/index.php?option=com_content&task=view&id=34&Itemid=61

I took a look at a few ones, but none seems to give all needed functionality.

You not only want to update the java stuff, but eventually add/remove files on the installation tree. You also want to have control of what version you have and
which version you're upgrading to. Last, but also eventually important, is to have a system that does not overwrite your modified files, just in case.

Truth is, I'd really like to use an of-the-shelf thing, but none seems to match all requirements so far, so eventually I'll pick up some base package and tune
it to our needs.

Best,
Álvaro Lopes
http://www.alvie.com/zpuino/


> = Mike
>
> On May 11, 2012, at 11:34 AM, Álvaro Lopes wrote:
>
>> Hi guys,
>>
>> You should know I do have a fork of Arduino IDE (for another CPU), and I kinda do most stuff focused on my system, and I tend to neglect Arduino often, also
>> because Arduino development is a bit strict, thus not allowing many changes from version to next version.
>>
>> Anyway, I'm facing a problem, and I do believe you are also facing a problem:
>>
>> There's no way do do "incremental" upgrades on Arduino system.
>>
>> Although release cycle in Arduino is large, it does not happen with my system, I've to constantly push changes to users, for the system, albeit stable,
>> sometimes requires changing at least the core libraries for size and performance sake.
>>
>> Right now, someone wishing to upgrade his Arduino installation (let's say, from "1.0" to upcoming "1.0.1") has do download a large zip file, remove the older
>> installation, and extract everything again. This presents a problem when you want to push small changes - being required to download a 100Mb zip file just
>> because there's one line changed on a ".h" is, well, stupid.
>>
>> So my idea is to build or integrate an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version (using
>> internet) to a more recent one (you can choose your next version), while eventually keeping your own "patched" files if you happened to mess with the IDE. Only
>> relevant files would be transferred.
>>
>> This is something that I will implement, and something I want to give back to Arduino community. I still have not figured out all requirements (server and
>> client), but should have done so in a week.
>>
>> How open are you to include this feature in some next release ?
>>
>> Best,
>> Álvaro Lopes
>> http://www.alvie.com/zpuino/
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

It sounds like you want a hybrid then; something Java-based to handle the Java pieces, and maybe git or similar to handle the toolchain pieces?

= Mike

On May 11, 2012, at 12:25 PM, �lvaro Lopes wrote:

> Michael Smith wrote:
>> There are a number of frameworks for this already; you might look at the (oft-maligned) WebStart, or something like http://www.interdirected.com/index.php?option=com_content&task=view&id=34&Itemid=61
>
> I took a look at a few ones, but none seems to give all needed functionality.
>
> You not only want to update the java stuff, but eventually add/remove files on the installation tree. You also want to have control of what version you have and
> which version you're upgrading to. Last, but also eventually important, is to have a system that does not overwrite your modified files, just in case.
>
> Truth is, I'd really like to use an of-the-shelf thing, but none seems to match all requirements so far, so eventually I'll pick up some base package and tune
> it to our needs.
>
> Best,
> �lvaro Lopes
> http://www.alvie.com/zpuino/
>
>
>> = Mike
>>
>> On May 11, 2012, at 11:34 AM, �lvaro Lopes wrote:
>>
>>> Hi guys,
>>>
>>> You should know I do have a fork of Arduino IDE (for another CPU), and I kinda do most stuff focused on my system, and I tend to neglect Arduino often, also
>>> because Arduino development is a bit strict, thus not allowing many changes from version to next version.
>>>
>>> Anyway, I'm facing a problem, and I do believe you are also facing a problem:
>>>
>>> There's no way do do "incremental" upgrades on Arduino system.
>>>
>>> Although release cycle in Arduino is large, it does not happen with my system, I've to constantly push changes to users, for the system, albeit stable,
>>> sometimes requires changing at least the core libraries for size and performance sake.
>>>
>>> Right now, someone wishing to upgrade his Arduino installation (let's say, from "1.0" to upcoming "1.0.1") has do download a large zip file, remove the older
>>> installation, and extract everything again. This presents a problem when you want to push small changes - being required to download a 100Mb zip file just
>>> because there's one line changed on a ".h" is, well, stupid.
>>>
>>> So my idea is to build or integrate an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version (using
>>> internet) to a more recent one (you can choose your next version), while eventually keeping your own "patched" files if you happened to mess with the IDE. Only
>>> relevant files would be transferred.
>>>
>>> This is something that I will implement, and something I want to give back to Arduino community. I still have not figured out all requirements (server and
>>> client), but should have done so in a week.
>>>
>>> How open are you to include this feature in some next release ?
>>>
>>> Best,
>>> �lvaro Lopes
>>> http://www.alvie.com/zpuino/
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

Michael Smith wrote:
> It sounds like you want a hybrid then; something Java-based to handle the Java pieces, and maybe git or similar to handle the toolchain pieces?

Something more like HTTP, so it's easier to deploy on server side. Something like minecraft updater, if you happen to play it. I think it's able to update all
tree, not only the JAR files.

I am still thinking about it, but I think an "extract" of a ZIP file on server, plus some metadata (like filesizes, dates, sha1 or similar) would do it, and
don't imply much effort for the deployer.

Álvaro Lopes
http://www.alvie.com/zpuino/

>
> = Mike
>
> On May 11, 2012, at 12:25 PM, Álvaro Lopes wrote:
>
>> Michael Smith wrote:
>>> There are a number of frameworks for this already; you might look at the (oft-maligned) WebStart, or something like http://www.interdirected.com/index.php?option=com_content&task=view&id=34&Itemid=61
>> I took a look at a few ones, but none seems to give all needed functionality.
>>
>> You not only want to update the java stuff, but eventually add/remove files on the installation tree. You also want to have control of what version you have and
>> which version you're upgrading to. Last, but also eventually important, is to have a system that does not overwrite your modified files, just in case.
>>
>> Truth is, I'd really like to use an of-the-shelf thing, but none seems to match all requirements so far, so eventually I'll pick up some base package and tune
>> it to our needs.
>>
>> Best,
>> Álvaro Lopes
>> http://www.alvie.com/zpuino/
>>
>>
>>> = Mike
>>>
>>> On May 11, 2012, at 11:34 AM, Álvaro Lopes wrote:
>>>
>>>> Hi guys,
>>>>
>>>> You should know I do have a fork of Arduino IDE (for another CPU), and I kinda do most stuff focused on my system, and I tend to neglect Arduino often, also
>>>> because Arduino development is a bit strict, thus not allowing many changes from version to next version.
>>>>
>>>> Anyway, I'm facing a problem, and I do believe you are also facing a problem:
>>>>
>>>> There's no way do do "incremental" upgrades on Arduino system.
>>>>
>>>> Although release cycle in Arduino is large, it does not happen with my system, I've to constantly push changes to users, for the system, albeit stable,
>>>> sometimes requires changing at least the core libraries for size and performance sake.
>>>>
>>>> Right now, someone wishing to upgrade his Arduino installation (let's say, from "1.0" to upcoming "1.0.1") has do download a large zip file, remove the older
>>>> installation, and extract everything again. This presents a problem when you want to push small changes - being required to download a 100Mb zip file just
>>>> because there's one line changed on a ".h" is, well, stupid.
>>>>
>>>> So my idea is to build or integrate an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version (using
>>>> internet) to a more recent one (you can choose your next version), while eventually keeping your own "patched" files if you happened to mess with the IDE. Only
>>>> relevant files would be transferred.
>>>>
>>>> This is something that I will implement, and something I want to give back to Arduino community. I still have not figured out all requirements (server and
>>>> client), but should have done so in a week.
>>>>
>>>> How open are you to include this feature in some next release ?
>>>>
>>>> Best,
>>>> Álvaro Lopes
>>>> http://www.alvie.com/zpuino/
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

An auto-updated would definitely be cool but, as with many things,
might be more complicated to maintain than it's worth. In particular,
I'm nervous about your desire to preserve files that have been locally
modified. That sounds like a recipe for things getting out of sync.
What happens when you've made local changes to a file that's been
updated in the new release? Or, more simply, what happens if only
part of the update finishes successfully and you end up with an
inconsistent state? I'm particularly wary about the fact that this
would be a custom implementation as this seems like kind of thing that
takes a lot of tweaking to get working consistently and reliably. I
don't want to discourage you from trying it but it might take some
work to convince me that it's ready for the standard distribution.

David

On Fri, May 11, 2012 at 2:34 PM, �lvaro Lopes <> wrote:
> Hi guys,
>
> You should know I do have a fork of Arduino IDE (for another CPU), and I kinda do most stuff focused on my system, and I tend to neglect Arduino often, also
> because Arduino development is a bit strict, thus not allowing many changes from version to next version.
>
> Anyway, I'm facing a problem, and I do believe you are also facing a problem:
>
> There's no way do do "incremental" upgrades on Arduino system.
>
> Although release cycle in Arduino is large, it does not happen with my system, I've to constantly push changes to users, for the system, albeit stable,
> sometimes requires changing at least the core libraries for size and performance sake.
>
> Right now, someone wishing to upgrade his Arduino installation (let's say, from "1.0" to upcoming "1.0.1") has do download a large zip file, remove the older
> installation, and extract everything again. This presents a problem when you want to push small changes - being required to download a 100Mb zip file just
> because there's one line changed on a ".h" is, well, stupid.
>
> So my idea is to build or integrate an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version (using
> internet) to a more recent one (you can choose your next version), while eventually keeping your own "patched" files if you happened to mess with the IDE. Only
> relevant files would be transferred.
>
> This is something that I will implement, and something I want to give back to Arduino community. I still have not figured out all requirements (server and
> client), but should have done so in a week.
>
> How open are you to include this feature in some next release ?
>
> Best,
> �lvaro Lopes
> http://www.alvie.com/zpuino/
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

On May 11, 2012, at 11:34 AM, �lvaro Lopes wrote:

> an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version

You could simply check for a source install (or have it configurable) and do the appropriate github commands.

Or maybe the Arduino project should be split up (IDE, tools, core) so that they can be updated individually.

It doesn't seem to me that it would be a worthwhile exercise to implement yet another Source Code Version Control System!

BillW


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

William "Chops" Westfield wrote:
> On May 11, 2012, at 11:34 AM, Álvaro Lopes wrote:
>
>> an updater system on Arduino IDE, something you can run from the menu and will incrementally upgrade your version
>
> You could simply check for a source install (or have it configurable) and do the appropriate github commands.

As long as you have all tools needed for that installed. Plus, we don't actually ship the binaries, but rather source code, so in addition you'd have to build
the system again.

> Or maybe the Arduino project should be split up (IDE, tools, core) so that they can be updated individually.

Think as the "upgrade" zip file being extracted in some web server, and you just download the ones that are actually different.

> It doesn't seem to me that it would be a worthwhile exercise to implement yet another Source Code Version Control System!

You cannot claim an updater is a SCVS. I am not focusing on source, but rather installation binaries here.

Best,
Álvaro

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

David A. Mellis wrote:
> An auto-updated would definitely be cool but, as with many things,
> might be more complicated to maintain than it's worth. In particular,
> I'm nervous about your desire to preserve files that have been locally
> modified. That sounds like a recipe for things getting out of sync.

Not necessarily, because the end user can opt for overwriting them or to keep old versions. Still, this might be considered an "advanced" functionality, off by
default (default is to overwrite everything).

> What happens when you've made local changes to a file that's been
> updated in the new release? Or, more simply, what happens if only
> part of the update finishes successfully and you end up with an
> inconsistent state?

You can get the two versions on disk, and then it's your responsibility to merge them. Again, opt for one, but have the other side by side.

> I'm particularly wary about the fact that this
> would be a custom implementation as this seems like kind of thing that
> takes a lot of tweaking to get working consistently and reliably. I
> don't want to discourage you from trying it but it might take some
> work to convince me that it's ready for the standard distribution.

I'm not saying that it is ready right now, but I'm to push it on my version (and that's for sure), which will probably take some time to stabilize. Only then
you can pick it and integrate it on standard distro.

There seems to be nothing OTS that provides this, otherwise I'd use it. Simpler. But unfortunately that's not the case.

Álvaro

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

David A. Mellis wrote:

> Or, more simply, what happens if only
> part of the update finishes successfully and you end up with an
> inconsistent state?

Clarification on this one: don't overwrite anything until all relevant parts are downloaded. Of course, as with any system, if you interrupt the process, weird
things might happen, but they also happen today: you're extracting the ZIP file and you lose power on the middle. So just reinstall or re-extract.

This, of course, will be taken in consideration like a whole lot of other factors. It's not the first time I write such a system, and the previous ones have
been very successful, in terms of stability and reliability.

Álvaro

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc