That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Same exact one. I just included it just in case.

--Rick

On Tue, Jan 4, 2011 at 9:01 PM, Tom Igoe <> wrote:
> Is this test suite any different than the one in the github repo?
>
> t.
>
> On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:
>
>> I just created a test and unpacked and tested it. I also created a
>> test based on Tom's code. Not sure I caught the maximum value error
>> for 0xFF.
>>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>>
>> I'm attaching the Arduino Test suite, and the test code so you can run
>> them as you debug. Let me know if this was helpful.
>>
>> --Rick
>>
>> Test run results:
>> Example:  ATS_StringAssignmentTest.pde
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringAssignmentTest
>> info.Free-memory     = 1110 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.378s
>>
>> FAILED (failures=1)
>>
>>
>> Example: ATS_StringMemoryTestPassByReference.pde
>>
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
>> info.Free-memory     = 1120 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.389s
>>
>> FAILED (failures=1)
>>
>>
>> Example: Tom Igoe Memory Test
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = Tom Igoe Memory Test
>> info.Free-memory     = 1328 bytes
>> Memory Usage (lost 9 bytes)                                 ... FAIL
>> --------------------------
>> Ran 1 tests in 1.374s
>>
>> FAILED (failures=1)
>>
>>
>>
>> --Rick
>>
>> Tom, rewrote your example as a test.
>>
>> //************************************************************************
>> //*   Arduino Tom Igoe Memory Test
>> //*           (C) 2010 by Rick Anderson
>> //*           Open source as per standard Arduino code
>> //*
>> //************************************************************************
>> //*   Jan 4,  2011     Started on Tom Igoe Memory Test
>> //************************************************************************
>>
>> #include      "WProgram.h"
>> #include      "HardwareSerial.h"
>> #include      
>>
>> //************************************************************************
>> void setup()
>> {
>>  int          startMemoryUsage;
>>  /*
>>    * Create variable for the tests.
>>   */
>>
>>  String webColor = "#000000";
>>
>>  int red = 1023/4;
>>  int blue = 512/4;
>>  int green = 128/4;
>>
>>  /*
>>  * initiate the test run
>>   */
>>  //Record starting memory amount
>>  startMemoryUsage     =       ATS_GetFreeMemory();
>>  //Begin test run
>>  ATS_begin("Arduino", "Tom Igoe Memory Test");
>>
>>  webColor = "#";
>>  webColor += String(red, HEX);
>>  webColor += String(green, HEX);
>>  webColor += String(blue, HEX);
>>
>>  ATS_ReportMemoryUsage(startMemoryUsage);
>>
>>  /*
>>   * Test complete
>>   */
>>
>>  ATS_end();
>> }
>>
>>
>> //************************************************************************
>> void loop()
>> {
>>
>>
>> }
>>
>>
>>
>>
>> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>>
>>> t.
>>>
>>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>>
>>>> That breaking of String, does it only occur with the SDM version?
>>>>
>>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>>
>>>>
>>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>>
>>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>>> Thanks, Alexander, and sorry for the delay.
>>>>
>>>> A few tests:
>>>>
>>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size.  However, when you start to do dynamic things, it does add up.  For example, the StringAppendOperator example is  4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>>
>>>> On some sketches it actually takes less memory.  For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.  However, it fails entirely on upload.  When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>>
>>>>
>>>> void setup() {
>>>>  Serial.begin(9600);
>>>>  Serial.println("Beginning");
>>>> }
>>>>
>>>> void loop() {
>>>>
>>>>  String webColor = "#";
>>>>
>>>>  int red = analogRead(A0)/4;
>>>>  webColor += String(red, HEX);
>>>>  int green = analogRead(A1)/4;
>>>>  webColor += String(green, HEX);
>>>>  int blue = analogRead(A2)/4;
>>>>
>>>>  webColor += String(blue, HEX);
>>>>  Serial.println(webColor);
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>>
>>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>>
>>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>>
>>>>>
>>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>>
>>>>>
>>>>> No error handling is implemented, what do you think we need?
>>>>>
>>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>>
>>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>>
>>>>>
>>>>> Hope to  get feedback from someone!
>>>>>
>>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>>
>>>>> I think you're right, but I also don't think that telling people to
>>>>> use the standard C library functions is a solution either.  Trying out
>>>>> this String class seems like one way of pushing forwards towards a
>>>>> compromise that provides both an easy-to-use API and a robust
>>>>> implementation.  For example, perhaps the buffer sizes need to be
>>>>> specified or provided when the String is instantiated.  Or we need an
>>>>> alternative implementation that does more reliable memory allocation
>>>>> (since we're not assuming that the internal buffer is continuous).
>>>>> Other ideas welcome.
>>>>>
>>>>> I think the problem is that the failure modes are not benign.  I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way?  Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>>
>>>>> --
>>>>> Alan Burlison
>>>>> --
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Same exact one. I just included it just in case.

--Rick

On Tue, Jan 4, 2011 at 9:01 PM, Tom Igoe <> wrote:
> Is this test suite any different than the one in the github repo?
>
> t.
>
> On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:
>
>> I just created a test and unpacked and tested it. I also created a
>> test based on Tom's code. Not sure I caught the maximum value error
>> for 0xFF.
>>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>>
>> I'm attaching the Arduino Test suite, and the test code so you can run
>> them as you debug. Let me know if this was helpful.
>>
>> --Rick
>>
>> Test run results:
>> Example:  ATS_StringAssignmentTest.pde
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringAssignmentTest
>> info.Free-memory     = 1110 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.378s
>>
>> FAILED (failures=1)
>>
>>
>> Example: ATS_StringMemoryTestPassByReference.pde
>>
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
>> info.Free-memory     = 1120 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.389s
>>
>> FAILED (failures=1)
>>
>>
>> Example: Tom Igoe Memory Test
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = Tom Igoe Memory Test
>> info.Free-memory     = 1328 bytes
>> Memory Usage (lost 9 bytes)                                 ... FAIL
>> --------------------------
>> Ran 1 tests in 1.374s
>>
>> FAILED (failures=1)
>>
>>
>>
>> --Rick
>>
>> Tom, rewrote your example as a test.
>>
>> //************************************************************************
>> //*   Arduino Tom Igoe Memory Test
>> //*           (C) 2010 by Rick Anderson
>> //*           Open source as per standard Arduino code
>> //*
>> //************************************************************************
>> //*   Jan 4,  2011     Started on Tom Igoe Memory Test
>> //************************************************************************
>>
>> #include      "WProgram.h"
>> #include      "HardwareSerial.h"
>> #include      
>>
>> //************************************************************************
>> void setup()
>> {
>>  int          startMemoryUsage;
>>  /*
>>    * Create variable for the tests.
>>   */
>>
>>  String webColor = "#000000";
>>
>>  int red = 1023/4;
>>  int blue = 512/4;
>>  int green = 128/4;
>>
>>  /*
>>  * initiate the test run
>>   */
>>  //Record starting memory amount
>>  startMemoryUsage     =       ATS_GetFreeMemory();
>>  //Begin test run
>>  ATS_begin("Arduino", "Tom Igoe Memory Test");
>>
>>  webColor = "#";
>>  webColor += String(red, HEX);
>>  webColor += String(green, HEX);
>>  webColor += String(blue, HEX);
>>
>>  ATS_ReportMemoryUsage(startMemoryUsage);
>>
>>  /*
>>   * Test complete
>>   */
>>
>>  ATS_end();
>> }
>>
>>
>> //************************************************************************
>> void loop()
>> {
>>
>>
>> }
>>
>>
>>
>>
>> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>>
>>> t.
>>>
>>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>>
>>>> That breaking of String, does it only occur with the SDM version?
>>>>
>>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>>
>>>>
>>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>>
>>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>>> Thanks, Alexander, and sorry for the delay.
>>>>
>>>> A few tests:
>>>>
>>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size.  However, when you start to do dynamic things, it does add up.  For example, the StringAppendOperator example is  4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>>
>>>> On some sketches it actually takes less memory.  For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.  However, it fails entirely on upload.  When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>>
>>>>
>>>> void setup() {
>>>>  Serial.begin(9600);
>>>>  Serial.println("Beginning");
>>>> }
>>>>
>>>> void loop() {
>>>>
>>>>  String webColor = "#";
>>>>
>>>>  int red = analogRead(A0)/4;
>>>>  webColor += String(red, HEX);
>>>>  int green = analogRead(A1)/4;
>>>>  webColor += String(green, HEX);
>>>>  int blue = analogRead(A2)/4;
>>>>
>>>>  webColor += String(blue, HEX);
>>>>  Serial.println(webColor);
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>>
>>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>>
>>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>>
>>>>>
>>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>>
>>>>>
>>>>> No error handling is implemented, what do you think we need?
>>>>>
>>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>>
>>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>>
>>>>>
>>>>> Hope to  get feedback from someone!
>>>>>
>>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>>
>>>>> I think you're right, but I also don't think that telling people to
>>>>> use the standard C library functions is a solution either.  Trying out
>>>>> this String class seems like one way of pushing forwards towards a
>>>>> compromise that provides both an easy-to-use API and a robust
>>>>> implementation.  For example, perhaps the buffer sizes need to be
>>>>> specified or provided when the String is instantiated.  Or we need an
>>>>> alternative implementation that does more reliable memory allocation
>>>>> (since we're not assuming that the internal buffer is continuous).
>>>>> Other ideas welcome.
>>>>>
>>>>> I think the problem is that the failure modes are not benign.  I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way?  Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>>
>>>>> --
>>>>> Alan Burlison
>>>>> --
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 04/01/2011 17:53, Rick Anderson wrote:

> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.

Most of the assumptions made by the test harness and test case code are
wrong, any reports of 'lost' memory are therefore likely to be wrong as
well. If you want to do leak detection you can't do it in the naive way
it is being done at present.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Same exact one. I just included it just in case.

--Rick

On Tue, Jan 4, 2011 at 9:01 PM, Tom Igoe <> wrote:
> Is this test suite any different than the one in the github repo?
>
> t.
>
> On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:
>
>> I just created a test and unpacked and tested it. I also created a
>> test based on Tom's code. Not sure I caught the maximum value error
>> for 0xFF.
>>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>>
>> I'm attaching the Arduino Test suite, and the test code so you can run
>> them as you debug. Let me know if this was helpful.
>>
>> --Rick
>>
>> Test run results:
>> Example:  ATS_StringAssignmentTest.pde
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringAssignmentTest
>> info.Free-memory     = 1110 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.378s
>>
>> FAILED (failures=1)
>>
>>
>> Example: ATS_StringMemoryTestPassByReference.pde
>>
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
>> info.Free-memory     = 1120 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.389s
>>
>> FAILED (failures=1)
>>
>>
>> Example: Tom Igoe Memory Test
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = Tom Igoe Memory Test
>> info.Free-memory     = 1328 bytes
>> Memory Usage (lost 9 bytes)                                 ... FAIL
>> --------------------------
>> Ran 1 tests in 1.374s
>>
>> FAILED (failures=1)
>>
>>
>>
>> --Rick
>>
>> Tom, rewrote your example as a test.
>>
>> //************************************************************************
>> //*   Arduino Tom Igoe Memory Test
>> //*           (C) 2010 by Rick Anderson
>> //*           Open source as per standard Arduino code
>> //*
>> //************************************************************************
>> //*   Jan 4,  2011     Started on Tom Igoe Memory Test
>> //************************************************************************
>>
>> #include      "WProgram.h"
>> #include      "HardwareSerial.h"
>> #include      
>>
>> //************************************************************************
>> void setup()
>> {
>>  int          startMemoryUsage;
>>  /*
>>    * Create variable for the tests.
>>   */
>>
>>  String webColor = "#000000";
>>
>>  int red = 1023/4;
>>  int blue = 512/4;
>>  int green = 128/4;
>>
>>  /*
>>  * initiate the test run
>>   */
>>  //Record starting memory amount
>>  startMemoryUsage     =       ATS_GetFreeMemory();
>>  //Begin test run
>>  ATS_begin("Arduino", "Tom Igoe Memory Test");
>>
>>  webColor = "#";
>>  webColor += String(red, HEX);
>>  webColor += String(green, HEX);
>>  webColor += String(blue, HEX);
>>
>>  ATS_ReportMemoryUsage(startMemoryUsage);
>>
>>  /*
>>   * Test complete
>>   */
>>
>>  ATS_end();
>> }
>>
>>
>> //************************************************************************
>> void loop()
>> {
>>
>>
>> }
>>
>>
>>
>>
>> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>>
>>> t.
>>>
>>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>>
>>>> That breaking of String, does it only occur with the SDM version?
>>>>
>>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>>
>>>>
>>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>>
>>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>>> Thanks, Alexander, and sorry for the delay.
>>>>
>>>> A few tests:
>>>>
>>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size.  However, when you start to do dynamic things, it does add up.  For example, the StringAppendOperator example is  4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>>
>>>> On some sketches it actually takes less memory.  For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.  However, it fails entirely on upload.  When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>>
>>>>
>>>> void setup() {
>>>>  Serial.begin(9600);
>>>>  Serial.println("Beginning");
>>>> }
>>>>
>>>> void loop() {
>>>>
>>>>  String webColor = "#";
>>>>
>>>>  int red = analogRead(A0)/4;
>>>>  webColor += String(red, HEX);
>>>>  int green = analogRead(A1)/4;
>>>>  webColor += String(green, HEX);
>>>>  int blue = analogRead(A2)/4;
>>>>
>>>>  webColor += String(blue, HEX);
>>>>  Serial.println(webColor);
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>>
>>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>>
>>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>>
>>>>>
>>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>>
>>>>>
>>>>> No error handling is implemented, what do you think we need?
>>>>>
>>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>>
>>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>>
>>>>>
>>>>> Hope to  get feedback from someone!
>>>>>
>>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>>
>>>>> I think you're right, but I also don't think that telling people to
>>>>> use the standard C library functions is a solution either.  Trying out
>>>>> this String class seems like one way of pushing forwards towards a
>>>>> compromise that provides both an easy-to-use API and a robust
>>>>> implementation.  For example, perhaps the buffer sizes need to be
>>>>> specified or provided when the String is instantiated.  Or we need an
>>>>> alternative implementation that does more reliable memory allocation
>>>>> (since we're not assuming that the internal buffer is continuous).
>>>>> Other ideas welcome.
>>>>>
>>>>> I think the problem is that the failure modes are not benign.  I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way?  Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>>
>>>>> --
>>>>> Alan Burlison
>>>>> --
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 04/01/2011 17:53, Rick Anderson wrote:

> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.

Most of the assumptions made by the test harness and test case code are
wrong, any reports of 'lost' memory are therefore likely to be wrong as
well. If you want to do leak detection you can't do it in the naive way
it is being done at present.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Alan,

The tests themselves are quick and dirty, and not strategic or
comprehensive. I'd like your input to create a good procedure for
memory testing. I also have feedback from Paul Stoffregen for the test
suite:
----
Another problem involves gcc's optimizer ignoring the inline request
for ATS_ReportMemoryUsage. As the comment in the code explains, it
must be inline to avoid measuring its own stack frame. I believe
adding this line in ArduinoTestSuite.h fixes the problem by forcing
gcc to inline.

inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
__attribute__((always_inline, unused));

The last problem is a tiny bit of dynamic memory permanently held for
the free list. To allow for this, the test really should call
malloc() and free() once, so the allocator is fully initialized before
testing.
----

I'd like to combine both sets of feedback, and set up some issues in
the system, then fix them. Is the above the problem you saw? Are their
other assumptions that would impact measuring memory? And updating the
memory test in the the test suite.

In terms of creating tests for memory leaks. It looks best to move all
the memory usage inside a function. What else could be done to
constrain the problem, and measuring more affectively.

I'm traveling back to NJ today, so I can't respond right away.

--Rick

On Wed, Jan 5, 2011 at 8:52 AM, Alan Burlison <> wrote:
> On 04/01/2011 17:53, Rick Anderson wrote:
>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>
> Most of the assumptions made by the test harness and test case code are
> wrong, any reports of 'lost' memory are therefore likely to be wrong as
> well.  If you want to do leak detection you can't do it in the naive way it
> is being done at present.
>
> --
> Alan Burlison
> --
>



--
President
Fair Use Building and Research (****) Labs
http://fubarlabs.org

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Same exact one. I just included it just in case.

--Rick

On Tue, Jan 4, 2011 at 9:01 PM, Tom Igoe <> wrote:
> Is this test suite any different than the one in the github repo?
>
> t.
>
> On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:
>
>> I just created a test and unpacked and tested it. I also created a
>> test based on Tom's code. Not sure I caught the maximum value error
>> for 0xFF.
>>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>>
>> I'm attaching the Arduino Test suite, and the test code so you can run
>> them as you debug. Let me know if this was helpful.
>>
>> --Rick
>>
>> Test run results:
>> Example:  ATS_StringAssignmentTest.pde
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringAssignmentTest
>> info.Free-memory     = 1110 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.378s
>>
>> FAILED (failures=1)
>>
>>
>> Example: ATS_StringMemoryTestPassByReference.pde
>>
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
>> info.Free-memory     = 1120 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.389s
>>
>> FAILED (failures=1)
>>
>>
>> Example: Tom Igoe Memory Test
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = Tom Igoe Memory Test
>> info.Free-memory     = 1328 bytes
>> Memory Usage (lost 9 bytes)                                 ... FAIL
>> --------------------------
>> Ran 1 tests in 1.374s
>>
>> FAILED (failures=1)
>>
>>
>>
>> --Rick
>>
>> Tom, rewrote your example as a test.
>>
>> //************************************************************************
>> //*   Arduino Tom Igoe Memory Test
>> //*           (C) 2010 by Rick Anderson
>> //*           Open source as per standard Arduino code
>> //*
>> //************************************************************************
>> //*   Jan 4,  2011     Started on Tom Igoe Memory Test
>> //************************************************************************
>>
>> #include      "WProgram.h"
>> #include      "HardwareSerial.h"
>> #include      
>>
>> //************************************************************************
>> void setup()
>> {
>>  int          startMemoryUsage;
>>  /*
>>    * Create variable for the tests.
>>   */
>>
>>  String webColor = "#000000";
>>
>>  int red = 1023/4;
>>  int blue = 512/4;
>>  int green = 128/4;
>>
>>  /*
>>  * initiate the test run
>>   */
>>  //Record starting memory amount
>>  startMemoryUsage     =       ATS_GetFreeMemory();
>>  //Begin test run
>>  ATS_begin("Arduino", "Tom Igoe Memory Test");
>>
>>  webColor = "#";
>>  webColor += String(red, HEX);
>>  webColor += String(green, HEX);
>>  webColor += String(blue, HEX);
>>
>>  ATS_ReportMemoryUsage(startMemoryUsage);
>>
>>  /*
>>   * Test complete
>>   */
>>
>>  ATS_end();
>> }
>>
>>
>> //************************************************************************
>> void loop()
>> {
>>
>>
>> }
>>
>>
>>
>>
>> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>>
>>> t.
>>>
>>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>>
>>>> That breaking of String, does it only occur with the SDM version?
>>>>
>>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>>
>>>>
>>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>>
>>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>>> Thanks, Alexander, and sorry for the delay.
>>>>
>>>> A few tests:
>>>>
>>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size.  However, when you start to do dynamic things, it does add up.  For example, the StringAppendOperator example is  4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>>
>>>> On some sketches it actually takes less memory.  For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.  However, it fails entirely on upload.  When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>>
>>>>
>>>> void setup() {
>>>>  Serial.begin(9600);
>>>>  Serial.println("Beginning");
>>>> }
>>>>
>>>> void loop() {
>>>>
>>>>  String webColor = "#";
>>>>
>>>>  int red = analogRead(A0)/4;
>>>>  webColor += String(red, HEX);
>>>>  int green = analogRead(A1)/4;
>>>>  webColor += String(green, HEX);
>>>>  int blue = analogRead(A2)/4;
>>>>
>>>>  webColor += String(blue, HEX);
>>>>  Serial.println(webColor);
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>>
>>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>>
>>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>>
>>>>>
>>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>>
>>>>>
>>>>> No error handling is implemented, what do you think we need?
>>>>>
>>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>>
>>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>>
>>>>>
>>>>> Hope to  get feedback from someone!
>>>>>
>>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>>
>>>>> I think you're right, but I also don't think that telling people to
>>>>> use the standard C library functions is a solution either.  Trying out
>>>>> this String class seems like one way of pushing forwards towards a
>>>>> compromise that provides both an easy-to-use API and a robust
>>>>> implementation.  For example, perhaps the buffer sizes need to be
>>>>> specified or provided when the String is instantiated.  Or we need an
>>>>> alternative implementation that does more reliable memory allocation
>>>>> (since we're not assuming that the internal buffer is continuous).
>>>>> Other ideas welcome.
>>>>>
>>>>> I think the problem is that the failure modes are not benign.  I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way?  Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>>
>>>>> --
>>>>> Alan Burlison
>>>>> --
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 04/01/2011 17:53, Rick Anderson wrote:

> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.

Most of the assumptions made by the test harness and test case code are
wrong, any reports of 'lost' memory are therefore likely to be wrong as
well. If you want to do leak detection you can't do it in the naive way
it is being done at present.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Alan,

The tests themselves are quick and dirty, and not strategic or
comprehensive. I'd like your input to create a good procedure for
memory testing. I also have feedback from Paul Stoffregen for the test
suite:
----
Another problem involves gcc's optimizer ignoring the inline request
for ATS_ReportMemoryUsage. As the comment in the code explains, it
must be inline to avoid measuring its own stack frame. I believe
adding this line in ArduinoTestSuite.h fixes the problem by forcing
gcc to inline.

inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
__attribute__((always_inline, unused));

The last problem is a tiny bit of dynamic memory permanently held for
the free list. To allow for this, the test really should call
malloc() and free() once, so the allocator is fully initialized before
testing.
----

I'd like to combine both sets of feedback, and set up some issues in
the system, then fix them. Is the above the problem you saw? Are their
other assumptions that would impact measuring memory? And updating the
memory test in the the test suite.

In terms of creating tests for memory leaks. It looks best to move all
the memory usage inside a function. What else could be done to
constrain the problem, and measuring more affectively.

I'm traveling back to NJ today, so I can't respond right away.

--Rick

On Wed, Jan 5, 2011 at 8:52 AM, Alan Burlison <> wrote:
> On 04/01/2011 17:53, Rick Anderson wrote:
>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>
> Most of the assumptions made by the test harness and test case code are
> wrong, any reports of 'lost' memory are therefore likely to be wrong as
> well.  If you want to do leak detection you can't do it in the naive way it
> is being done at present.
>
> --
> Alan Burlison
> --
>



--
President
Fair Use Building and Research (****) Labs
http://fubarlabs.org

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 05/01/2011 15:19, Rick Anderson wrote:

> ----
> Another problem involves gcc's optimizer ignoring the inline request
> for ATS_ReportMemoryUsage. As the comment in the code explains, it
> must be inline to avoid measuring its own stack frame. I believe
> adding this line in ArduinoTestSuite.h fixes the problem by forcing
> gcc to inline.
>
> inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
> __attribute__((always_inline, unused));

Yes, that's one of the problems. And even the __attribute__
'workaround' is a fragile hack.

> The last problem is a tiny bit of dynamic memory permanently held for
> the free list. To allow for this, the test really should call
> malloc() and free() once, so the allocator is fully initialized before

That's part of a problem, but an initial call malloc/free call isn't
guaranteed to make all the issues go away either. From looking at the
avr-libc malloc implementation it does try to coalesce the heap, but
relying on multiple malloc implementation artefacts is a really bad idea
- there's no guarantee that future malloc implementations will always
coalesce on every free, and there's no guarantee that __bss_end or
__brkval will always be adjusted on each malloc/free call.

The other issue is that of object lifetimes. The test code has
instances of String objects which are allocated on the stack and that
are still in scope when the 'leak' memory check is performed. That
means the String destructors won't have been called and the dynamically
allocated part of the String will still be allocated.

A true memory leak is a block of allocated memory that has no references
to it. However that's not what the test code is checking for. If a
library routine used malloc to perform a one-time library-private buffer
allocation where the buffer is retained for the lifetime of the program,
the current leak checking would incorrectly report that as a leak. It
isn't a leak because there's still a reference to the block inside the
library.

The only semi-reliable way to do what you want is to use an instrumented
malloc implementation, and that's going to be very difficult to
implement on a constrained platform such as the AVR. If you want to
test the String implementation for leaks, my advice is to do it on a
more capable platform (e.g. *nix) and use one of the many instrumented
mallocs that are available.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Same exact one. I just included it just in case.

--Rick

On Tue, Jan 4, 2011 at 9:01 PM, Tom Igoe <> wrote:
> Is this test suite any different than the one in the github repo?
>
> t.
>
> On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:
>
>> I just created a test and unpacked and tested it. I also created a
>> test based on Tom's code. Not sure I caught the maximum value error
>> for 0xFF.
>>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>>
>> I'm attaching the Arduino Test suite, and the test code so you can run
>> them as you debug. Let me know if this was helpful.
>>
>> --Rick
>>
>> Test run results:
>> Example:  ATS_StringAssignmentTest.pde
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringAssignmentTest
>> info.Free-memory     = 1110 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.378s
>>
>> FAILED (failures=1)
>>
>>
>> Example: ATS_StringMemoryTestPassByReference.pde
>>
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
>> info.Free-memory     = 1120 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.389s
>>
>> FAILED (failures=1)
>>
>>
>> Example: Tom Igoe Memory Test
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = Tom Igoe Memory Test
>> info.Free-memory     = 1328 bytes
>> Memory Usage (lost 9 bytes)                                 ... FAIL
>> --------------------------
>> Ran 1 tests in 1.374s
>>
>> FAILED (failures=1)
>>
>>
>>
>> --Rick
>>
>> Tom, rewrote your example as a test.
>>
>> //************************************************************************
>> //*   Arduino Tom Igoe Memory Test
>> //*           (C) 2010 by Rick Anderson
>> //*           Open source as per standard Arduino code
>> //*
>> //************************************************************************
>> //*   Jan 4,  2011     Started on Tom Igoe Memory Test
>> //************************************************************************
>>
>> #include      "WProgram.h"
>> #include      "HardwareSerial.h"
>> #include      
>>
>> //************************************************************************
>> void setup()
>> {
>>  int          startMemoryUsage;
>>  /*
>>    * Create variable for the tests.
>>   */
>>
>>  String webColor = "#000000";
>>
>>  int red = 1023/4;
>>  int blue = 512/4;
>>  int green = 128/4;
>>
>>  /*
>>  * initiate the test run
>>   */
>>  //Record starting memory amount
>>  startMemoryUsage     =       ATS_GetFreeMemory();
>>  //Begin test run
>>  ATS_begin("Arduino", "Tom Igoe Memory Test");
>>
>>  webColor = "#";
>>  webColor += String(red, HEX);
>>  webColor += String(green, HEX);
>>  webColor += String(blue, HEX);
>>
>>  ATS_ReportMemoryUsage(startMemoryUsage);
>>
>>  /*
>>   * Test complete
>>   */
>>
>>  ATS_end();
>> }
>>
>>
>> //************************************************************************
>> void loop()
>> {
>>
>>
>> }
>>
>>
>>
>>
>> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>>
>>> t.
>>>
>>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>>
>>>> That breaking of String, does it only occur with the SDM version?
>>>>
>>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>>
>>>>
>>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>>
>>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>>> Thanks, Alexander, and sorry for the delay.
>>>>
>>>> A few tests:
>>>>
>>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size.  However, when you start to do dynamic things, it does add up.  For example, the StringAppendOperator example is  4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>>
>>>> On some sketches it actually takes less memory.  For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.  However, it fails entirely on upload.  When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>>
>>>>
>>>> void setup() {
>>>>  Serial.begin(9600);
>>>>  Serial.println("Beginning");
>>>> }
>>>>
>>>> void loop() {
>>>>
>>>>  String webColor = "#";
>>>>
>>>>  int red = analogRead(A0)/4;
>>>>  webColor += String(red, HEX);
>>>>  int green = analogRead(A1)/4;
>>>>  webColor += String(green, HEX);
>>>>  int blue = analogRead(A2)/4;
>>>>
>>>>  webColor += String(blue, HEX);
>>>>  Serial.println(webColor);
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>>
>>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>>
>>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>>
>>>>>
>>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>>
>>>>>
>>>>> No error handling is implemented, what do you think we need?
>>>>>
>>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>>
>>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>>
>>>>>
>>>>> Hope to  get feedback from someone!
>>>>>
>>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>>
>>>>> I think you're right, but I also don't think that telling people to
>>>>> use the standard C library functions is a solution either.  Trying out
>>>>> this String class seems like one way of pushing forwards towards a
>>>>> compromise that provides both an easy-to-use API and a robust
>>>>> implementation.  For example, perhaps the buffer sizes need to be
>>>>> specified or provided when the String is instantiated.  Or we need an
>>>>> alternative implementation that does more reliable memory allocation
>>>>> (since we're not assuming that the internal buffer is continuous).
>>>>> Other ideas welcome.
>>>>>
>>>>> I think the problem is that the failure modes are not benign.  I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way?  Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>>
>>>>> --
>>>>> Alan Burlison
>>>>> --
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 04/01/2011 17:53, Rick Anderson wrote:

> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.

Most of the assumptions made by the test harness and test case code are
wrong, any reports of 'lost' memory are therefore likely to be wrong as
well. If you want to do leak detection you can't do it in the naive way
it is being done at present.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Alan,

The tests themselves are quick and dirty, and not strategic or
comprehensive. I'd like your input to create a good procedure for
memory testing. I also have feedback from Paul Stoffregen for the test
suite:
----
Another problem involves gcc's optimizer ignoring the inline request
for ATS_ReportMemoryUsage. As the comment in the code explains, it
must be inline to avoid measuring its own stack frame. I believe
adding this line in ArduinoTestSuite.h fixes the problem by forcing
gcc to inline.

inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
__attribute__((always_inline, unused));

The last problem is a tiny bit of dynamic memory permanently held for
the free list. To allow for this, the test really should call
malloc() and free() once, so the allocator is fully initialized before
testing.
----

I'd like to combine both sets of feedback, and set up some issues in
the system, then fix them. Is the above the problem you saw? Are their
other assumptions that would impact measuring memory? And updating the
memory test in the the test suite.

In terms of creating tests for memory leaks. It looks best to move all
the memory usage inside a function. What else could be done to
constrain the problem, and measuring more affectively.

I'm traveling back to NJ today, so I can't respond right away.

--Rick

On Wed, Jan 5, 2011 at 8:52 AM, Alan Burlison <> wrote:
> On 04/01/2011 17:53, Rick Anderson wrote:
>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>
> Most of the assumptions made by the test harness and test case code are
> wrong, any reports of 'lost' memory are therefore likely to be wrong as
> well.  If you want to do leak detection you can't do it in the naive way it
> is being done at present.
>
> --
> Alan Burlison
> --
>



--
President
Fair Use Building and Research (****) Labs
http://fubarlabs.org

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 05/01/2011 15:19, Rick Anderson wrote:

> ----
> Another problem involves gcc's optimizer ignoring the inline request
> for ATS_ReportMemoryUsage. As the comment in the code explains, it
> must be inline to avoid measuring its own stack frame. I believe
> adding this line in ArduinoTestSuite.h fixes the problem by forcing
> gcc to inline.
>
> inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
> __attribute__((always_inline, unused));

Yes, that's one of the problems. And even the __attribute__
'workaround' is a fragile hack.

> The last problem is a tiny bit of dynamic memory permanently held for
> the free list. To allow for this, the test really should call
> malloc() and free() once, so the allocator is fully initialized before

That's part of a problem, but an initial call malloc/free call isn't
guaranteed to make all the issues go away either. From looking at the
avr-libc malloc implementation it does try to coalesce the heap, but
relying on multiple malloc implementation artefacts is a really bad idea
- there's no guarantee that future malloc implementations will always
coalesce on every free, and there's no guarantee that __bss_end or
__brkval will always be adjusted on each malloc/free call.

The other issue is that of object lifetimes. The test code has
instances of String objects which are allocated on the stack and that
are still in scope when the 'leak' memory check is performed. That
means the String destructors won't have been called and the dynamically
allocated part of the String will still be allocated.

A true memory leak is a block of allocated memory that has no references
to it. However that's not what the test code is checking for. If a
library routine used malloc to perform a one-time library-private buffer
allocation where the buffer is retained for the lifetime of the program,
the current leak checking would incorrectly report that as a leak. It
isn't a leak because there's still a reference to the block inside the
library.

The only semi-reliable way to do what you want is to use an instrumented
malloc implementation, and that's going to be very difficult to
implement on a constrained platform such as the AVR. If you want to
test the String implementation for leaks, my advice is to do it on a
more capable platform (e.g. *nix) and use one of the many instrumented
mallocs that are available.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Slightly unrelated to what you guys are discussing now but:
I'm not sure if I've made this clear, but the new version of String that I'm
working on now does not use malloc or any other c function for dynamic
memory. What happens is that I allocate memory statically on the stack and
serve that memory dynamically to the instances of String.

I have some time on my hands now and have a look at getting the += not to
break the code

On Wed, Jan 5, 2011 at 7:10 PM, Alan Burlison <>wrote:

> On 05/01/2011 15:19, Rick Anderson wrote:
>
> ----
>> Another problem involves gcc's optimizer ignoring the inline request
>> for ATS_ReportMemoryUsage. As the comment in the code explains, it
>> must be inline to avoid measuring its own stack frame. I believe
>> adding this line in ArduinoTestSuite.h fixes the problem by forcing
>> gcc to inline.
>>
>> inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
>> __attribute__((always_inline, unused));
>>
>
> Yes, that's one of the problems. And even the __attribute__ 'workaround'
> is a fragile hack.
>
>
> The last problem is a tiny bit of dynamic memory permanently held for
>> the free list. To allow for this, the test really should call
>> malloc() and free() once, so the allocator is fully initialized before
>>
>
> That's part of a problem, but an initial call malloc/free call isn't
> guaranteed to make all the issues go away either. From looking at the
> avr-libc malloc implementation it does try to coalesce the heap, but relying
> on multiple malloc implementation artefacts is a really bad idea - there's
> no guarantee that future malloc implementations will always coalesce on
> every free, and there's no guarantee that __bss_end or __brkval will always
> be adjusted on each malloc/free call.
>
> The other issue is that of object lifetimes. The test code has instances
> of String objects which are allocated on the stack and that are still in
> scope when the 'leak' memory check is performed. That means the String
> destructors won't have been called and the dynamically allocated part of the
> String will still be allocated.
>
> A true memory leak is a block of allocated memory that has no references to
> it. However that's not what the test code is checking for. If a library
> routine used malloc to perform a one-time library-private buffer allocation
> where the buffer is retained for the lifetime of the program, the current
> leak checking would incorrectly report that as a leak. It isn't a leak
> because there's still a reference to the block inside the library.
>
> The only semi-reliable way to do what you want is to use an instrumented
> malloc implementation, and that's going to be very difficult to implement on
> a constrained platform such as the AVR. If you want to test the String
> implementation for leaks, my advice is to do it on a more capable platform
> (e.g. *nix) and use one of the many instrumented mallocs that are available.
>
> --
> Alan Burlison
> --
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Same exact one. I just included it just in case.

--Rick

On Tue, Jan 4, 2011 at 9:01 PM, Tom Igoe <> wrote:
> Is this test suite any different than the one in the github repo?
>
> t.
>
> On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:
>
>> I just created a test and unpacked and tested it. I also created a
>> test based on Tom's code. Not sure I caught the maximum value error
>> for 0xFF.
>>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>>
>> I'm attaching the Arduino Test suite, and the test code so you can run
>> them as you debug. Let me know if this was helpful.
>>
>> --Rick
>>
>> Test run results:
>> Example:  ATS_StringAssignmentTest.pde
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringAssignmentTest
>> info.Free-memory     = 1110 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.378s
>>
>> FAILED (failures=1)
>>
>>
>> Example: ATS_StringMemoryTestPassByReference.pde
>>
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
>> info.Free-memory     = 1120 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.389s
>>
>> FAILED (failures=1)
>>
>>
>> Example: Tom Igoe Memory Test
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = Tom Igoe Memory Test
>> info.Free-memory     = 1328 bytes
>> Memory Usage (lost 9 bytes)                                 ... FAIL
>> --------------------------
>> Ran 1 tests in 1.374s
>>
>> FAILED (failures=1)
>>
>>
>>
>> --Rick
>>
>> Tom, rewrote your example as a test.
>>
>> //************************************************************************
>> //*   Arduino Tom Igoe Memory Test
>> //*           (C) 2010 by Rick Anderson
>> //*           Open source as per standard Arduino code
>> //*
>> //************************************************************************
>> //*   Jan 4,  2011     Started on Tom Igoe Memory Test
>> //************************************************************************
>>
>> #include      "WProgram.h"
>> #include      "HardwareSerial.h"
>> #include      
>>
>> //************************************************************************
>> void setup()
>> {
>>  int          startMemoryUsage;
>>  /*
>>    * Create variable for the tests.
>>   */
>>
>>  String webColor = "#000000";
>>
>>  int red = 1023/4;
>>  int blue = 512/4;
>>  int green = 128/4;
>>
>>  /*
>>  * initiate the test run
>>   */
>>  //Record starting memory amount
>>  startMemoryUsage     =       ATS_GetFreeMemory();
>>  //Begin test run
>>  ATS_begin("Arduino", "Tom Igoe Memory Test");
>>
>>  webColor = "#";
>>  webColor += String(red, HEX);
>>  webColor += String(green, HEX);
>>  webColor += String(blue, HEX);
>>
>>  ATS_ReportMemoryUsage(startMemoryUsage);
>>
>>  /*
>>   * Test complete
>>   */
>>
>>  ATS_end();
>> }
>>
>>
>> //************************************************************************
>> void loop()
>> {
>>
>>
>> }
>>
>>
>>
>>
>> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>>
>>> t.
>>>
>>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>>
>>>> That breaking of String, does it only occur with the SDM version?
>>>>
>>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>>
>>>>
>>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>>
>>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>>> Thanks, Alexander, and sorry for the delay.
>>>>
>>>> A few tests:
>>>>
>>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size.  However, when you start to do dynamic things, it does add up.  For example, the StringAppendOperator example is  4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>>
>>>> On some sketches it actually takes less memory.  For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.  However, it fails entirely on upload.  When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>>
>>>>
>>>> void setup() {
>>>>  Serial.begin(9600);
>>>>  Serial.println("Beginning");
>>>> }
>>>>
>>>> void loop() {
>>>>
>>>>  String webColor = "#";
>>>>
>>>>  int red = analogRead(A0)/4;
>>>>  webColor += String(red, HEX);
>>>>  int green = analogRead(A1)/4;
>>>>  webColor += String(green, HEX);
>>>>  int blue = analogRead(A2)/4;
>>>>
>>>>  webColor += String(blue, HEX);
>>>>  Serial.println(webColor);
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>>
>>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>>
>>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>>
>>>>>
>>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>>
>>>>>
>>>>> No error handling is implemented, what do you think we need?
>>>>>
>>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>>
>>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>>
>>>>>
>>>>> Hope to  get feedback from someone!
>>>>>
>>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>>
>>>>> I think you're right, but I also don't think that telling people to
>>>>> use the standard C library functions is a solution either.  Trying out
>>>>> this String class seems like one way of pushing forwards towards a
>>>>> compromise that provides both an easy-to-use API and a robust
>>>>> implementation.  For example, perhaps the buffer sizes need to be
>>>>> specified or provided when the String is instantiated.  Or we need an
>>>>> alternative implementation that does more reliable memory allocation
>>>>> (since we're not assuming that the internal buffer is continuous).
>>>>> Other ideas welcome.
>>>>>
>>>>> I think the problem is that the failure modes are not benign.  I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way?  Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>>
>>>>> --
>>>>> Alan Burlison
>>>>> --
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 04/01/2011 17:53, Rick Anderson wrote:

> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.

Most of the assumptions made by the test harness and test case code are
wrong, any reports of 'lost' memory are therefore likely to be wrong as
well. If you want to do leak detection you can't do it in the naive way
it is being done at present.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Alan,

The tests themselves are quick and dirty, and not strategic or
comprehensive. I'd like your input to create a good procedure for
memory testing. I also have feedback from Paul Stoffregen for the test
suite:
----
Another problem involves gcc's optimizer ignoring the inline request
for ATS_ReportMemoryUsage. As the comment in the code explains, it
must be inline to avoid measuring its own stack frame. I believe
adding this line in ArduinoTestSuite.h fixes the problem by forcing
gcc to inline.

inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
__attribute__((always_inline, unused));

The last problem is a tiny bit of dynamic memory permanently held for
the free list. To allow for this, the test really should call
malloc() and free() once, so the allocator is fully initialized before
testing.
----

I'd like to combine both sets of feedback, and set up some issues in
the system, then fix them. Is the above the problem you saw? Are their
other assumptions that would impact measuring memory? And updating the
memory test in the the test suite.

In terms of creating tests for memory leaks. It looks best to move all
the memory usage inside a function. What else could be done to
constrain the problem, and measuring more affectively.

I'm traveling back to NJ today, so I can't respond right away.

--Rick

On Wed, Jan 5, 2011 at 8:52 AM, Alan Burlison <> wrote:
> On 04/01/2011 17:53, Rick Anderson wrote:
>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>
> Most of the assumptions made by the test harness and test case code are
> wrong, any reports of 'lost' memory are therefore likely to be wrong as
> well.  If you want to do leak detection you can't do it in the naive way it
> is being done at present.
>
> --
> Alan Burlison
> --
>



--
President
Fair Use Building and Research (****) Labs
http://fubarlabs.org

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 05/01/2011 15:19, Rick Anderson wrote:

> ----
> Another problem involves gcc's optimizer ignoring the inline request
> for ATS_ReportMemoryUsage. As the comment in the code explains, it
> must be inline to avoid measuring its own stack frame. I believe
> adding this line in ArduinoTestSuite.h fixes the problem by forcing
> gcc to inline.
>
> inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
> __attribute__((always_inline, unused));

Yes, that's one of the problems. And even the __attribute__
'workaround' is a fragile hack.

> The last problem is a tiny bit of dynamic memory permanently held for
> the free list. To allow for this, the test really should call
> malloc() and free() once, so the allocator is fully initialized before

That's part of a problem, but an initial call malloc/free call isn't
guaranteed to make all the issues go away either. From looking at the
avr-libc malloc implementation it does try to coalesce the heap, but
relying on multiple malloc implementation artefacts is a really bad idea
- there's no guarantee that future malloc implementations will always
coalesce on every free, and there's no guarantee that __bss_end or
__brkval will always be adjusted on each malloc/free call.

The other issue is that of object lifetimes. The test code has
instances of String objects which are allocated on the stack and that
are still in scope when the 'leak' memory check is performed. That
means the String destructors won't have been called and the dynamically
allocated part of the String will still be allocated.

A true memory leak is a block of allocated memory that has no references
to it. However that's not what the test code is checking for. If a
library routine used malloc to perform a one-time library-private buffer
allocation where the buffer is retained for the lifetime of the program,
the current leak checking would incorrectly report that as a leak. It
isn't a leak because there's still a reference to the block inside the
library.

The only semi-reliable way to do what you want is to use an instrumented
malloc implementation, and that's going to be very difficult to
implement on a constrained platform such as the AVR. If you want to
test the String implementation for leaks, my advice is to do it on a
more capable platform (e.g. *nix) and use one of the many instrumented
mallocs that are available.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Slightly unrelated to what you guys are discussing now but:
I'm not sure if I've made this clear, but the new version of String that I'm
working on now does not use malloc or any other c function for dynamic
memory. What happens is that I allocate memory statically on the stack and
serve that memory dynamically to the instances of String.

I have some time on my hands now and have a look at getting the += not to
break the code

On Wed, Jan 5, 2011 at 7:10 PM, Alan Burlison <>wrote:

> On 05/01/2011 15:19, Rick Anderson wrote:
>
> ----
>> Another problem involves gcc's optimizer ignoring the inline request
>> for ATS_ReportMemoryUsage. As the comment in the code explains, it
>> must be inline to avoid measuring its own stack frame. I believe
>> adding this line in ArduinoTestSuite.h fixes the problem by forcing
>> gcc to inline.
>>
>> inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
>> __attribute__((always_inline, unused));
>>
>
> Yes, that's one of the problems. And even the __attribute__ 'workaround'
> is a fragile hack.
>
>
> The last problem is a tiny bit of dynamic memory permanently held for
>> the free list. To allow for this, the test really should call
>> malloc() and free() once, so the allocator is fully initialized before
>>
>
> That's part of a problem, but an initial call malloc/free call isn't
> guaranteed to make all the issues go away either. From looking at the
> avr-libc malloc implementation it does try to coalesce the heap, but relying
> on multiple malloc implementation artefacts is a really bad idea - there's
> no guarantee that future malloc implementations will always coalesce on
> every free, and there's no guarantee that __bss_end or __brkval will always
> be adjusted on each malloc/free call.
>
> The other issue is that of object lifetimes. The test code has instances
> of String objects which are allocated on the stack and that are still in
> scope when the 'leak' memory check is performed. That means the String
> destructors won't have been called and the dynamically allocated part of the
> String will still be allocated.
>
> A true memory leak is a block of allocated memory that has no references to
> it. However that's not what the test code is checking for. If a library
> routine used malloc to perform a one-time library-private buffer allocation
> where the buffer is retained for the lifetime of the program, the current
> leak checking would incorrectly report that as a leak. It isn't a leak
> because there's still a reference to the block inside the library.
>
> The only semi-reliable way to do what you want is to use an instrumented
> malloc implementation, and that's going to be very difficult to implement on
> a constrained platform such as the AVR. If you want to test the String
> implementation for leaks, my advice is to do it on a more capable platform
> (e.g. *nix) and use one of the many instrumented mallocs that are available.
>
> --
> Alan Burlison
> --
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 05/01/2011 18:25, Alexander Brevig wrote:

> Slightly unrelated to what you guys are discussing now but:
> I'm not sure if I've made this clear, but the new version of String that I'm
> working on now does not use malloc or any other c function for dynamic
> memory. What happens is that I allocate memory statically on the stack and
> serve that memory dynamically to the instances of String.

Does that mean that the total space occupied by all Strings has to be
decided up front? If so, what is the behaviour when an allocation fails?

To be honest I don't really see the point of doing this - all you are
doing is re-implementing malloc.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

That breaking of String, does it only occur with the SDM version?

If not, did you find any way to break it? Maybe any of the old faults
related to memory?


I actually have no time to check this myself right now, so I opted to just
ask you.

On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:

> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below
> is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.
> However, it fails entirely on upload. When the value of (analog 0)/4 is
> high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I
> think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <>
> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't
> been through all the use cases so this might not work, but perhaps using a
> statically-allocated string ("NOMEM" ?) as a sentinel value instead of null
> when malloc fails might be one way? Also, there might be some benefit in
> making String immutable a la Java - overloaded operators in C++ have a
> tendency to generate anonymous temporaries, which is going to lead to heap
> fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.

t.

On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:

> That breaking of String, does it only occur with the SDM version?
>
> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>
>
> I actually have no time to check this myself right now, so I opted to just ask you.
>
> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> Thanks, Alexander, and sorry for the delay.
>
> A few tests:
>
> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>
> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>
>
> void setup() {
> Serial.begin(9600);
> Serial.println("Beginning");
> }
>
> void loop() {
>
> String webColor = "#";
>
> int red = analogRead(A0)/4;
> webColor += String(red, HEX);
> int green = analogRead(A1)/4;
> webColor += String(green, HEX);
> int blue = analogRead(A2)/4;
>
> webColor += String(blue, HEX);
> Serial.println(webColor);
> }
>
>
>
>
>
> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>
> > Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
> > Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
> >
> > Now you're ready to test the modified String library, that does not fragment the heap.
> >
> >
> > In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
> >
> >
> > No error handling is implemented, what do you think we need?
> >
> > I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
> >
> > I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
> >
> >
> > Hope to get feedback from someone!
> >
> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
> > On 15/12/2010 16:19, David A. Mellis wrote:
> >
> > I think you're right, but I also don't think that telling people to
> > use the standard C library functions is a solution either. Trying out
> > this String class seems like one way of pushing forwards towards a
> > compromise that provides both an easy-to-use API and a robust
> > implementation. For example, perhaps the buffer sizes need to be
> > specified or provided when the String is instantiated. Or we need an
> > alternative implementation that does more reliable memory allocation
> > (since we're not assuming that the internal buffer is continuous).
> > Other ideas welcome.
> >
> > I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
> >
> > --
> > Alan Burlison
> > --
> >
> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. There seems to be a bug with the assignment operator, I'm on it and I'll get
back to you as soon as I get something done.

Thank you for your tests and feedback.


I'll do some tests on my own before I repost it here.

On Tue, Jan 4, 2011 at 6:53 PM, Rick Anderson
<>wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
> > It broke in the non-SDM version too, but it took longer to break. In the
> SDM version it breaks immediately. So it looks like whatever you did
> highlights the faults sooner, which is good. It also points to a need for an
> error response of some sort, in answer to your question last mail.
> >
> > t.
> >
> > On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
> >
> >> That breaking of String, does it only occur with the SDM version?
> >>
> >> If not, did you find any way to break it? Maybe any of the old faults
> related to memory?
> >>
> >>
> >> I actually have no time to check this myself right now, so I opted to
> just ask you.
> >>
> >> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
> >> Thanks, Alexander, and sorry for the delay.
> >>
> >> A few tests:
> >>
> >> For code that doesn't change the strings much, there's not a big
> difference, a couple dozen bytes' difference in code size. However, when
> you start to do dynamic things, it does add up. For example, the
> StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the
> StaticDynamicMemory version. So it does start to take its toll, memory-wise.
> >>
> >> On some sketches it actually takes less memory. For example, the one
> below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in
> 0022. However, it fails entirely on upload. When the value of (analog 0)/4
> is high (0xFF), it never gets out of the setup(). So there's still a pretty
> easy way to break String, even with these improvements.
> >>
> >>
> >> void setup() {
> >> Serial.begin(9600);
> >> Serial.println("Beginning");
> >> }
> >>
> >> void loop() {
> >>
> >> String webColor = "#";
> >>
> >> int red = analogRead(A0)/4;
> >> webColor += String(red, HEX);
> >> int green = analogRead(A1)/4;
> >> webColor += String(green, HEX);
> >> int blue = analogRead(A2)/4;
> >>
> >> webColor += String(blue, HEX);
> >> Serial.println(webColor);
> >> }
> >>
> >>
> >>
> >>
> >>
> >> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
> >>
> >> > Attached is a zip containing WString.h and .cpp plus
> StaticDynamicMemory.h.
> >> > Take a backup of the original WString files, and extract this zip into
> the hardware/arduino/cores/arduino folder.
> >> >
> >> > Now you're ready to test the modified String library, that does not
> fragment the heap.
> >> >
> >> >
> >> > In the WString header there is a #define called STRING_BUFFER which is
> set to 100 for now, change this to suit your test needs.
> >> >
> >> >
> >> > No error handling is implemented, what do you think we need?
> >> >
> >> > I plan to add an availableMemory():int method to the SDM library, and
> maybe some error flags and checks like hasError():int error
> >> >
> >> > I also suspect that this library is a bit slower performance wise, but
> I think that's just a cost we'll have to pay to being able to play with
> dynamic strings.
> >> >
> >> >
> >> > Hope to get feedback from someone!
> >> >
> >> > On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <
> > wrote:
> >> > On 15/12/2010 16:19, David A. Mellis wrote:
> >> >
> >> > I think you're right, but I also don't think that telling people to
> >> > use the standard C library functions is a solution either. Trying out
> >> > this String class seems like one way of pushing forwards towards a
> >> > compromise that provides both an easy-to-use API and a robust
> >> > implementation. For example, perhaps the buffer sizes need to be
> >> > specified or provided when the String is instantiated. Or we need an
> >> > alternative implementation that does more reliable memory allocation
> >> > (since we're not assuming that the internal buffer is continuous).
> >> > Other ideas welcome.
> >> >
> >> > I think the problem is that the failure modes are not benign. I
> haven't been through all the use cases so this might not work, but perhaps
> using a statically-allocated string ("NOMEM" ?) as a sentinel value instead
> of null when malloc fails might be one way? Also, there might be some
> benefit in making String immutable a la Java - overloaded operators in C++
> have a tendency to generate anonymous temporaries, which is going to lead to
> heap fragmentation as well.
> >> >
> >> > --
> >> > Alan Burlison
> >> > --
> >> >
> >> > _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Is this test suite any different than the one in the github repo?

t.

On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:

> I just created a test and unpacked and tested it. I also created a
> test based on Tom's code. Not sure I caught the maximum value error
> for 0xFF.
>
> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.
>
> I'm attaching the Arduino Test suite, and the test code so you can run
> them as you debug. Let me know if this was helpful.
>
> --Rick
>
> Test run results:
> Example: ATS_StringAssignmentTest.pde
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringAssignmentTest
> info.Free-memory = 1110 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.378s
>
> FAILED (failures=1)
>
>
> Example: ATS_StringMemoryTestPassByReference.pde
>
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
> info.Free-memory = 1120 bytes
> Memory Usage (lost 59 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.389s
>
> FAILED (failures=1)
>
>
> Example: Tom Igoe Memory Test
> info.MANUFACTURER = Arduino
> info.CPU-NAME = ATmega328P
> info.GCC-Version = 4.3.2
> info.AVR-LibC-Ver = 1.6.4
> info.Compiled-date = Jan 4 2011
> info.Test-Suite-Name = Tom Igoe Memory Test
> info.Free-memory = 1328 bytes
> Memory Usage (lost 9 bytes) ... FAIL
> --------------------------
> Ran 1 tests in 1.374s
>
> FAILED (failures=1)
>
>
>
> --Rick
>
> Tom, rewrote your example as a test.
>
> //************************************************************************
> //* Arduino Tom Igoe Memory Test
> //* (C) 2010 by Rick Anderson
> //* Open source as per standard Arduino code
> //*
> //************************************************************************
> //* Jan 4, 2011 Started on Tom Igoe Memory Test
> //************************************************************************
>
> #include "WProgram.h"
> #include "HardwareSerial.h"
> #include
>
> //************************************************************************
> void setup()
> {
> int startMemoryUsage;
> /*
> * Create variable for the tests.
> */
>
> String webColor = "#000000";
>
> int red = 1023/4;
> int blue = 512/4;
> int green = 128/4;
>
> /*
> * initiate the test run
> */
> //Record starting memory amount
> startMemoryUsage = ATS_GetFreeMemory();
> //Begin test run
> ATS_begin("Arduino", "Tom Igoe Memory Test");
>
> webColor = "#";
> webColor += String(red, HEX);
> webColor += String(green, HEX);
> webColor += String(blue, HEX);
>
> ATS_ReportMemoryUsage(startMemoryUsage);
>
> /*
> * Test complete
> */
>
> ATS_end();
> }
>
>
> //************************************************************************
> void loop()
> {
>
>
> }
>
>
>
>
> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>
>> t.
>>
>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>
>>> That breaking of String, does it only occur with the SDM version?
>>>
>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>
>>>
>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>
>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>> Thanks, Alexander, and sorry for the delay.
>>>
>>> A few tests:
>>>
>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size. However, when you start to do dynamic things, it does add up. For example, the StringAppendOperator example is 4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>
>>> On some sketches it actually takes less memory. For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022. However, it fails entirely on upload. When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>
>>>
>>> void setup() {
>>> Serial.begin(9600);
>>> Serial.println("Beginning");
>>> }
>>>
>>> void loop() {
>>>
>>> String webColor = "#";
>>>
>>> int red = analogRead(A0)/4;
>>> webColor += String(red, HEX);
>>> int green = analogRead(A1)/4;
>>> webColor += String(green, HEX);
>>> int blue = analogRead(A2)/4;
>>>
>>> webColor += String(blue, HEX);
>>> Serial.println(webColor);
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>
>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>
>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>
>>>>
>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>
>>>>
>>>> No error handling is implemented, what do you think we need?
>>>>
>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>
>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>
>>>>
>>>> Hope to get feedback from someone!
>>>>
>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>
>>>> I think you're right, but I also don't think that telling people to
>>>> use the standard C library functions is a solution either. Trying out
>>>> this String class seems like one way of pushing forwards towards a
>>>> compromise that provides both an easy-to-use API and a robust
>>>> implementation. For example, perhaps the buffer sizes need to be
>>>> specified or provided when the String is instantiated. Or we need an
>>>> alternative implementation that does more reliable memory allocation
>>>> (since we're not assuming that the internal buffer is continuous).
>>>> Other ideas welcome.
>>>>
>>>> I think the problem is that the failure modes are not benign. I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way? Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>
>>>> --
>>>> Alan Burlison
>>>> --
>>>>
>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Same exact one. I just included it just in case.

--Rick

On Tue, Jan 4, 2011 at 9:01 PM, Tom Igoe <> wrote:
> Is this test suite any different than the one in the github repo?
>
> t.
>
> On Jan 4, 2011, at 12:53 PM, Rick Anderson wrote:
>
>> I just created a test and unpacked and tested it. I also created a
>> test based on Tom's code. Not sure I caught the maximum value error
>> for 0xFF.
>>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>>
>> I'm attaching the Arduino Test suite, and the test code so you can run
>> them as you debug. Let me know if this was helpful.
>>
>> --Rick
>>
>> Test run results:
>> Example:  ATS_StringAssignmentTest.pde
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringAssignmentTest
>> info.Free-memory     = 1110 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.378s
>>
>> FAILED (failures=1)
>>
>>
>> Example: ATS_StringMemoryTestPassByReference.pde
>>
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = ATS_StringMemoryTestPassByReference
>> info.Free-memory     = 1120 bytes
>> Memory Usage (lost 59 bytes)                                ... FAIL
>> --------------------------
>> Ran 1 tests in 1.389s
>>
>> FAILED (failures=1)
>>
>>
>> Example: Tom Igoe Memory Test
>> info.MANUFACTURER    = Arduino
>> info.CPU-NAME        = ATmega328P
>> info.GCC-Version     = 4.3.2
>> info.AVR-LibC-Ver    = 1.6.4
>> info.Compiled-date   = Jan  4 2011
>> info.Test-Suite-Name = Tom Igoe Memory Test
>> info.Free-memory     = 1328 bytes
>> Memory Usage (lost 9 bytes)                                 ... FAIL
>> --------------------------
>> Ran 1 tests in 1.374s
>>
>> FAILED (failures=1)
>>
>>
>>
>> --Rick
>>
>> Tom, rewrote your example as a test.
>>
>> //************************************************************************
>> //*   Arduino Tom Igoe Memory Test
>> //*           (C) 2010 by Rick Anderson
>> //*           Open source as per standard Arduino code
>> //*
>> //************************************************************************
>> //*   Jan 4,  2011     Started on Tom Igoe Memory Test
>> //************************************************************************
>>
>> #include      "WProgram.h"
>> #include      "HardwareSerial.h"
>> #include      
>>
>> //************************************************************************
>> void setup()
>> {
>>  int          startMemoryUsage;
>>  /*
>>    * Create variable for the tests.
>>   */
>>
>>  String webColor = "#000000";
>>
>>  int red = 1023/4;
>>  int blue = 512/4;
>>  int green = 128/4;
>>
>>  /*
>>  * initiate the test run
>>   */
>>  //Record starting memory amount
>>  startMemoryUsage     =       ATS_GetFreeMemory();
>>  //Begin test run
>>  ATS_begin("Arduino", "Tom Igoe Memory Test");
>>
>>  webColor = "#";
>>  webColor += String(red, HEX);
>>  webColor += String(green, HEX);
>>  webColor += String(blue, HEX);
>>
>>  ATS_ReportMemoryUsage(startMemoryUsage);
>>
>>  /*
>>   * Test complete
>>   */
>>
>>  ATS_end();
>> }
>>
>>
>> //************************************************************************
>> void loop()
>> {
>>
>>
>> }
>>
>>
>>
>>
>> On Mon, Jan 3, 2011 at 9:35 PM, Tom Igoe <> wrote:
>>> It broke in the non-SDM version too, but it took longer to break. In the SDM version it breaks immediately. So it looks like whatever you did highlights the faults sooner, which is good. It also points to a need for an error response of some sort, in answer to your question last mail.
>>>
>>> t.
>>>
>>> On Jan 3, 2011, at 4:44 PM, Alexander Brevig wrote:
>>>
>>>> That breaking of String, does it only occur with the SDM version?
>>>>
>>>> If not, did you find any way to break it? Maybe any of the old faults related to memory?
>>>>
>>>>
>>>> I actually have no time to check this myself right now, so I opted to just ask you.
>>>>
>>>> On Mon, Jan 3, 2011 at 10:35 PM, Tom Igoe <> wrote:
>>>> Thanks, Alexander, and sorry for the delay.
>>>>
>>>> A few tests:
>>>>
>>>> For code that doesn't change the strings much, there's not a big difference, a couple dozen bytes' difference in code size.  However, when you start to do dynamic things, it does add up.  For example, the StringAppendOperator example is  4910 bytes in 0022, and 5148 bytes with the StaticDynamicMemory version. So it does start to take its toll, memory-wise.
>>>>
>>>> On some sketches it actually takes less memory.  For example, the one below is 3738 bytes using the StaticDynamicMemory version, and 3986 bytes in 0022.  However, it fails entirely on upload.  When the value of (analog 0)/4 is high (0xFF), it never gets out of the setup(). So there's still a pretty easy way to break String, even with these improvements.
>>>>
>>>>
>>>> void setup() {
>>>>  Serial.begin(9600);
>>>>  Serial.println("Beginning");
>>>> }
>>>>
>>>> void loop() {
>>>>
>>>>  String webColor = "#";
>>>>
>>>>  int red = analogRead(A0)/4;
>>>>  webColor += String(red, HEX);
>>>>  int green = analogRead(A1)/4;
>>>>  webColor += String(green, HEX);
>>>>  int blue = analogRead(A2)/4;
>>>>
>>>>  webColor += String(blue, HEX);
>>>>  Serial.println(webColor);
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 16, 2010, at 1:27 AM, Alexander Brevig wrote:
>>>>
>>>>> Attached is a zip containing WString.h and .cpp plus StaticDynamicMemory.h.
>>>>> Take a backup of the original WString files, and extract this zip into the hardware/arduino/cores/arduino folder.
>>>>>
>>>>> Now you're ready to test the modified String library, that does not fragment the heap.
>>>>>
>>>>>
>>>>> In the WString header there is a #define called STRING_BUFFER which is set to 100 for now, change this to suit your test needs.
>>>>>
>>>>>
>>>>> No error handling is implemented, what do you think we need?
>>>>>
>>>>> I plan to add an availableMemory():int method to the SDM library, and maybe some error flags and checks like hasError():int error
>>>>>
>>>>> I also suspect that this library is a bit slower performance wise, but I think that's just a cost we'll have to pay to being able to play with dynamic strings.
>>>>>
>>>>>
>>>>> Hope to  get feedback from someone!
>>>>>
>>>>> On Thu, Dec 16, 2010 at 1:11 AM, Alan Burlison <> wrote:
>>>>> On 15/12/2010 16:19, David A. Mellis wrote:
>>>>>
>>>>> I think you're right, but I also don't think that telling people to
>>>>> use the standard C library functions is a solution either.  Trying out
>>>>> this String class seems like one way of pushing forwards towards a
>>>>> compromise that provides both an easy-to-use API and a robust
>>>>> implementation.  For example, perhaps the buffer sizes need to be
>>>>> specified or provided when the String is instantiated.  Or we need an
>>>>> alternative implementation that does more reliable memory allocation
>>>>> (since we're not assuming that the internal buffer is continuous).
>>>>> Other ideas welcome.
>>>>>
>>>>> I think the problem is that the failure modes are not benign.  I haven't been through all the use cases so this might not work, but perhaps using a statically-allocated string ("NOMEM" ?) as a sentinel value instead of null when malloc fails might be one way?  Also, there might be some benefit in making String immutable a la Java - overloaded operators in C++ have a tendency to generate anonymous temporaries, which is going to lead to heap fragmentation as well.
>>>>>
>>>>> --
>>>>> Alan Burlison
>>>>> --
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 04/01/2011 17:53, Rick Anderson wrote:

> I also ran the tests for bug 411. All the tests compile, but there is
> still memory loss.

Most of the assumptions made by the test harness and test case code are
wrong, any reports of 'lost' memory are therefore likely to be wrong as
well. If you want to do leak detection you can't do it in the naive way
it is being done at present.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Alan,

The tests themselves are quick and dirty, and not strategic or
comprehensive. I'd like your input to create a good procedure for
memory testing. I also have feedback from Paul Stoffregen for the test
suite:
----
Another problem involves gcc's optimizer ignoring the inline request
for ATS_ReportMemoryUsage. As the comment in the code explains, it
must be inline to avoid measuring its own stack frame. I believe
adding this line in ArduinoTestSuite.h fixes the problem by forcing
gcc to inline.

inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
__attribute__((always_inline, unused));

The last problem is a tiny bit of dynamic memory permanently held for
the free list. To allow for this, the test really should call
malloc() and free() once, so the allocator is fully initialized before
testing.
----

I'd like to combine both sets of feedback, and set up some issues in
the system, then fix them. Is the above the problem you saw? Are their
other assumptions that would impact measuring memory? And updating the
memory test in the the test suite.

In terms of creating tests for memory leaks. It looks best to move all
the memory usage inside a function. What else could be done to
constrain the problem, and measuring more affectively.

I'm traveling back to NJ today, so I can't respond right away.

--Rick

On Wed, Jan 5, 2011 at 8:52 AM, Alan Burlison <> wrote:
> On 04/01/2011 17:53, Rick Anderson wrote:
>
>> I also ran the tests for bug 411. All the tests compile, but there is
>> still memory loss.
>
> Most of the assumptions made by the test harness and test case code are
> wrong, any reports of 'lost' memory are therefore likely to be wrong as
> well.  If you want to do leak detection you can't do it in the naive way it
> is being done at present.
>
> --
> Alan Burlison
> --
>



--
President
Fair Use Building and Research (****) Labs
http://fubarlabs.org

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 05/01/2011 15:19, Rick Anderson wrote:

> ----
> Another problem involves gcc's optimizer ignoring the inline request
> for ATS_ReportMemoryUsage. As the comment in the code explains, it
> must be inline to avoid measuring its own stack frame. I believe
> adding this line in ArduinoTestSuite.h fixes the problem by forcing
> gcc to inline.
>
> inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
> __attribute__((always_inline, unused));

Yes, that's one of the problems. And even the __attribute__
'workaround' is a fragile hack.

> The last problem is a tiny bit of dynamic memory permanently held for
> the free list. To allow for this, the test really should call
> malloc() and free() once, so the allocator is fully initialized before

That's part of a problem, but an initial call malloc/free call isn't
guaranteed to make all the issues go away either. From looking at the
avr-libc malloc implementation it does try to coalesce the heap, but
relying on multiple malloc implementation artefacts is a really bad idea
- there's no guarantee that future malloc implementations will always
coalesce on every free, and there's no guarantee that __bss_end or
__brkval will always be adjusted on each malloc/free call.

The other issue is that of object lifetimes. The test code has
instances of String objects which are allocated on the stack and that
are still in scope when the 'leak' memory check is performed. That
means the String destructors won't have been called and the dynamically
allocated part of the String will still be allocated.

A true memory leak is a block of allocated memory that has no references
to it. However that's not what the test code is checking for. If a
library routine used malloc to perform a one-time library-private buffer
allocation where the buffer is retained for the lifetime of the program,
the current leak checking would incorrectly report that as a leak. It
isn't a leak because there's still a reference to the block inside the
library.

The only semi-reliable way to do what you want is to use an instrumented
malloc implementation, and that's going to be very difficult to
implement on a constrained platform such as the AVR. If you want to
test the String implementation for leaks, my advice is to do it on a
more capable platform (e.g. *nix) and use one of the many instrumented
mallocs that are available.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Slightly unrelated to what you guys are discussing now but:
I'm not sure if I've made this clear, but the new version of String that I'm
working on now does not use malloc or any other c function for dynamic
memory. What happens is that I allocate memory statically on the stack and
serve that memory dynamically to the instances of String.

I have some time on my hands now and have a look at getting the += not to
break the code

On Wed, Jan 5, 2011 at 7:10 PM, Alan Burlison <>wrote:

> On 05/01/2011 15:19, Rick Anderson wrote:
>
> ----
>> Another problem involves gcc's optimizer ignoring the inline request
>> for ATS_ReportMemoryUsage. As the comment in the code explains, it
>> must be inline to avoid measuring its own stack frame. I believe
>> adding this line in ArduinoTestSuite.h fixes the problem by forcing
>> gcc to inline.
>>
>> inline void ATS_ReportMemoryUsage(int _memoryUsageAtStart)
>> __attribute__((always_inline, unused));
>>
>
> Yes, that's one of the problems. And even the __attribute__ 'workaround'
> is a fragile hack.
>
>
> The last problem is a tiny bit of dynamic memory permanently held for
>> the free list. To allow for this, the test really should call
>> malloc() and free() once, so the allocator is fully initialized before
>>
>
> That's part of a problem, but an initial call malloc/free call isn't
> guaranteed to make all the issues go away either. From looking at the
> avr-libc malloc implementation it does try to coalesce the heap, but relying
> on multiple malloc implementation artefacts is a really bad idea - there's
> no guarantee that future malloc implementations will always coalesce on
> every free, and there's no guarantee that __bss_end or __brkval will always
> be adjusted on each malloc/free call.
>
> The other issue is that of object lifetimes. The test code has instances
> of String objects which are allocated on the stack and that are still in
> scope when the 'leak' memory check is performed. That means the String
> destructors won't have been called and the dynamically allocated part of the
> String will still be allocated.
>
> A true memory leak is a block of allocated memory that has no references to
> it. However that's not what the test code is checking for. If a library
> routine used malloc to perform a one-time library-private buffer allocation
> where the buffer is retained for the lifetime of the program, the current
> leak checking would incorrectly report that as a leak. It isn't a leak
> because there's still a reference to the block inside the library.
>
> The only semi-reliable way to do what you want is to use an instrumented
> malloc implementation, and that's going to be very difficult to implement on
> a constrained platform such as the AVR. If you want to test the String
> implementation for leaks, my advice is to do it on a more capable platform
> (e.g. *nix) and use one of the many instrumented mallocs that are available.
>
> --
> Alan Burlison
> --
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. On 05/01/2011 18:25, Alexander Brevig wrote:

> Slightly unrelated to what you guys are discussing now but:
> I'm not sure if I've made this clear, but the new version of String that I'm
> working on now does not use malloc or any other c function for dynamic
> memory. What happens is that I allocate memory statically on the stack and
> serve that memory dynamically to the instances of String.

Does that mean that the total space occupied by all Strings has to be
decided up front? If so, what is the behaviour when an allocation fails?

To be honest I don't really see the point of doing this - all you are
doing is re-implementing malloc.

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. No, actually I am not reimplementing malloc, because my algorithm guarantees
no fragmentation and always 100% effective use of the allocated memory.

"Does that mean that the total space occupied by all Strings has to be
decided up front? "
Yes, just like the buffer for the serial ++

I know I would opt to have the restriction for a predetermined allocated
space WITH error reporting (how we do is up to you guys, feed me with ideas
I'll implement whatever we agree is best) than having
seemingly infinite memory that crashes and burns.

On Wed, Jan 5, 2011 at 7:33 PM, Alan Burlison <>wrote:

> On 05/01/2011 18:25, Alexander Brevig wrote:
>
> Slightly unrelated to what you guys are discussing now but:
>> I'm not sure if I've made this clear, but the new version of String that
>> I'm
>> working on now does not use malloc or any other c function for dynamic
>> memory. What happens is that I allocate memory statically on the stack and
>> serve that memory dynamically to the instances of String.
>>
>
> Does that mean that the total space occupied by all Strings has to be
> decided up front? If so, what is the behaviour when an allocation fails?
>
> To be honest I don't really see the point of doing this - all you are doing
> is re-implementing malloc.
>
> --
> Alan Burlison
> --
>