I;m working on a number of examples using Strings to do fancy things like customize web services from an ethernet shield, and finding need for some sgtring expansion. I'd like feedback on the best solutions for these:

The print ASCII conversions would be really useful in Strings. For example:

String webColor = "#";
webColor += (analogRead(0)/4, HEX);
webColor += (analogRead(1)/4, HEX);
webColor += (analogRead(2)/4, HEX);

Which makes more sense: giving String access to Print.h, or adding a new function like toString(number, BASE)?

Second, I'm working with the SD library, and realizing that it would be really handy if any function that took a char array as input also took a string. For example:

File myFile = SD.open("index.html");
Would be nice if you could do this:

String fileName = "bla";
// dynamically change fileName in your code
myFile = SD.open(fileName);

Currently the toCharArray and getBytes aren't working,so that rules using them out.

Thoughts?
_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

Tom,

My only concern is that unless your're lucky or really careful the
String library can cause weird problems. I'd like to isolate all those
situations in tests, but I have create a bunch of small tests in
separate sketches to cover all the conditions. If you run into in
problems or think of a test case let me know. I'd like to have a
comprehensive test suite for String.

--Rick

On Tue, Dec 14, 2010 at 12:17 PM, Tom Igoe <> wrote:
> I;m working on a number of examples using Strings to do fancy things like customize web services from an ethernet shield, and finding need for some sgtring expansion. I'd like feedback on the best solutions for these:
>
> The print ASCII conversions would be really useful in Strings. �For example:
>
> String webColor = "#";
> webColor += (analogRead(0)/4, HEX);
> webColor += (analogRead(1)/4, HEX);
> webColor += (analogRead(2)/4, HEX);
>
> Which makes more sense: �giving String access to Print.h, or adding a new function like toString(number, BASE)?
>
> Second, I'm working with the SD library, and realizing that it would be really handy if any function that took a char array as input also took a string. �For example:
>
> File myFile = SD.open("index.html");
> Would be nice if you could do this:
>
> String fileName = "bla";
> // dynamically change fileName in your code
> �myFile = SD.open(fileName);
>
> Currently the toCharArray and getBytes aren't working,so that rules using them out.
>
> Thoughts?
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

Well, there's the case I laid out below. I can think of others. Whether you're dealing with the web or mail or other application layer net apps, it's generally all ASCII. Conversion functions would be very useful.

With all of these, I am using String as a way to deal with character arrays without having to expose pointers to the user.

Case 1:
Let's say you have a bunch of files on an SD card and you want to serve them as a simple server using the Ethernet shield. You want to modify some of the content based on sensor readings. You need to take the files in to an array (and String makes this easy and clear, even for non-C programmers), convert the sensor readings to a string, replace a substring with the readings, and spit them out to the client.

Case 2: Assume case 1, but you're using the sensor readings to set a web color, which is in hex.

Case 3: you want to take in a realtime clock reading and serve it. You need to do timezone conversions on it too.

Is that the kind of case you want?

t.

On Dec 14, 2010, at 1:25 PM, Rick Anderson wrote:

> Tom,
>
> My only concern is that unless your're lucky or really careful the
> String library can cause weird problems. I'd like to isolate all those
> situations in tests, but I have create a bunch of small tests in
> separate sketches to cover all the conditions. If you run into in
> problems or think of a test case let me know. I'd like to have a
> comprehensive test suite for String.
>
> --Rick
>
> On Tue, Dec 14, 2010 at 12:17 PM, Tom Igoe <> wrote:
>> I;m working on a number of examples using Strings to do fancy things like customize web services from an ethernet shield, and finding need for some sgtring expansion. I'd like feedback on the best solutions for these:
>>
>> The print ASCII conversions would be really useful in Strings. For example:
>>
>> String webColor = "#";
>> webColor += (analogRead(0)/4, HEX);
>> webColor += (analogRead(1)/4, HEX);
>> webColor += (analogRead(2)/4, HEX);
>>
>> Which makes more sense: giving String access to Print.h, or adding a new function like toString(number, BASE)?
>>
>> Second, I'm working with the SD library, and realizing that it would be really handy if any function that took a char array as input also took a string. For example:
>>
>> File myFile = SD.open("index.html");
>> Would be nice if you could do this:
>>
>> String fileName = "bla";
>> // dynamically change fileName in your code
>> myFile = SD.open(fileName);
>>
>> Currently the toCharArray and getBytes aren't working,so that rules using them out.
>>
>> Thoughts?
>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

The String constructor can take a base parameter, so you should be
able to do: s += String(123, HEX). The toCharArray() and getBytes()
functions should work, you just need to pass in a buffer to copy the
characters to (and its length). I'd like to add support for String
filename in the SD library, but haven't had a chance yet.

On Tue, Dec 14, 2010 at 12:17 PM, Tom Igoe <> wrote:
> I;m working on a number of examples using Strings to do fancy things like customize web services from an ethernet shield, and finding need for some sgtring expansion. I'd like feedback on the best solutions for these:
>
> The print ASCII conversions would be really useful in Strings. �For example:
>
> String webColor = "#";
> webColor += (analogRead(0)/4, HEX);
> webColor += (analogRead(1)/4, HEX);
> webColor += (analogRead(2)/4, HEX);
>
> Which makes more sense: �giving String access to Print.h, or adding a new function like toString(number, BASE)?
>
> Second, I'm working with the SD library, and realizing that it would be really handy if any function that took a char array as input also took a string. �For example:
>
> File myFile = SD.open("index.html");
> Would be nice if you could do this:
>
> String fileName = "bla";
> // dynamically change fileName in your code
> �myFile = SD.open(fileName);
>
> Currently the toCharArray and getBytes aren't working,so that rules using them out.
>
> Thoughts?
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

Last time I tried the getBytes() and toCharArray() they didn't compile, but that was before your candidate release. I'll try this, thanks, I should have thought of that.

On Dec 14, 2010, at 1:56 PM, David A. Mellis wrote:

> The String constructor can take a base parameter, so you should be
> able to do: s += String(123, HEX). The toCharArray() and getBytes()
> functions should work, you just need to pass in a buffer to copy the
> characters to (and its length). I'd like to add support for String
> filename in the SD library, but haven't had a chance yet.
>
> On Tue, Dec 14, 2010 at 12:17 PM, Tom Igoe <> wrote:
>> I;m working on a number of examples using Strings to do fancy things like customize web services from an ethernet shield, and finding need for some sgtring expansion. I'd like feedback on the best solutions for these:
>>
>> The print ASCII conversions would be really useful in Strings. For example:
>>
>> String webColor = "#";
>> webColor += (analogRead(0)/4, HEX);
>> webColor += (analogRead(1)/4, HEX);
>> webColor += (analogRead(2)/4, HEX);
>>
>> Which makes more sense: giving String access to Print.h, or adding a new function like toString(number, BASE)?
>>
>> Second, I'm working with the SD library, and realizing that it would be really handy if any function that took a char array as input also took a string. For example:
>>
>> File myFile = SD.open("index.html");
>> Would be nice if you could do this:
>>
>> String fileName = "bla";
>> // dynamically change fileName in your code
>> myFile = SD.open(fileName);
>>
>> Currently the toCharArray and getBytes aren't working,so that rules using them out.
>>
>> Thoughts?
>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

Here's a potentially interesting test case for String, Rick (or anyone else interested):

The code below fails under certain conditions. Specifically, when red hits about 0x33, it freezes up. A typical last reading before freeze is #33fffc. More interesting is what happens when red's about 0x11, see the results below. Definitely looks like it's reading memory it shouldn't beyond the String length. Furthermore, when it freezes, a simple reset won't fix it. It needs to be re-uploaded to get started again.

Simple fix for the problem: Make the initial String longer. Same code where String webColor = "bgcolor=#"; works fine. But it clearly seems like there's an unpleasant memory leak still to be had.

void setup() {
Serial.begin(9600);
Serial.println("Beginning");
}

void loop() {

String webColor = "#";

int red = analogRead(A0)/4;
webColor += String(red, HEX);
int green = analogRead(A1)/4;
webColor += String(green, HEX);
int blue = analogRead(A2)/4;

webColor += String(blue, HEX);
Serial.println(webColor);
}


Results:

#11fffc
!��V��+��5
#11fffc
#11ffff
#11fffd
#12fffe
#11fffe
#11fffc
#11ffff
#11fffb
#11ffff
#11fffc
#11ffff
#11fffe
#11fffd�!��V��+��5
#11fffc
#11ffff
#11fffd
#11fffe
#11fffe
#11fffc
#11ffff
#11fffb
#11ffff
#11fffc
#11ffff
#11fffe
#12fffd�!��V��+��5




_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

Tom,

Thanks outlining those cases. It's good to work this out. One
principal is that we need to test the String Library, and SD Card
library as stand alone library. Then when the two are used together we
may not need as many test cases for cross usages.

Case 1:
String Library tests:
Read strings into an array
convert values to string
replace substring
print strings

SD Library test:
Create arbitrary number of files on an SD Card
Read list of files from an SD Card

Ethernet Shield Test:
Get string values
Serve string values

Existing Arduino Test Suite:
Includes tests of sensor readings analog and digital

Say those tests are in place and the system passes. How much
integration testing would you need? Would we need to build that
combination and test that specific situation?
In that case the test would need to test really the integration of the devices.

Case 2:
If the tests for String libraries pass, and the SD card libraries
pass, Ethernet libraries pass, then this test may not be necessary.

Case 3:
Sensor readings are tested in the Arduino General Test
Create test that the real time clock values are correct.
Is there a timezone library? If you use a c/c++ library then we don't
have to test it. If there is an Arduino Timezone library then we would
have to test that.

I hope my logic is making sense. Let me know. The basic idea is if we
have the test coverage in place, about basic libraries and features,
then integration tests only have to show that combination of devices
are functional.

--Rick

On Tue, Dec 14, 2010 at 1:48 PM, Tom Igoe <> wrote:
> Well, there's the case I laid out below. �I can think of others. �Whether you're dealing with the web or mail or other application layer net apps, it's generally all ASCII. �Conversion functions would be very useful.
>
> With all of these, I am using String as a way to deal with character arrays without having to expose pointers to the user.
>
> Case 1:
> Let's say you have a bunch of files on an SD card and you want to serve them as a simple server using the Ethernet shield. You want to modify some of the content based on sensor readings. You need to take the files in to an array (and String makes this easy and clear, even for non-C programmers), convert the sensor readings to a string, replace a substring with the readings, and �spit them out to the client.
>
> Case 2: �Assume case 1, but you're using the sensor readings to set a web color, which is in hex.
>
> Case 3: �you want to take in a realtime clock reading and serve it. �You need to do timezone conversions on it too.
>
> Is that the kind of case you want?
>
> t.
>
> On Dec 14, 2010, at 1:25 PM, Rick Anderson wrote:
>
>> Tom,
>>
>> My only concern is that unless your're lucky or really careful the
>> String library can cause weird problems. I'd like to isolate all those
>> situations in tests, but I have create a bunch of small tests in
>> separate sketches to cover all the conditions. If you run into in
>> problems or think of �a test case let me know. I'd like to have a
>> comprehensive test suite for String.
>>
>> --Rick
>>
>> On Tue, Dec 14, 2010 at 12:17 PM, Tom Igoe <> wrote:
>>> I;m working on a number of examples using Strings to do fancy things like customize web services from an ethernet shield, and finding need for some sgtring expansion. I'd like feedback on the best solutions for these:
>>>
>>> The print ASCII conversions would be really useful in Strings. �For example:
>>>
>>> String webColor = "#";
>>> webColor += (analogRead(0)/4, HEX);
>>> webColor += (analogRead(1)/4, HEX);
>>> webColor += (analogRead(2)/4, HEX);
>>>
>>> Which makes more sense: �giving String access to Print.h, or adding a new function like toString(number, BASE)?
>>>
>>> Second, I'm working with the SD library, and realizing that it would be really handy if any function that took a char array as input also took a string. �For example:
>>>
>>> File myFile = SD.open("index.html");
>>> Would be nice if you could do this:
>>>
>>> String fileName = "bla";
>>> // dynamically change fileName in your code
>>> �myFile = SD.open(fileName);
>>>
>>> Currently the toCharArray and getBytes aren't working,so that rules using them out.
>>>
>>> Thoughts?
>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

On 14/12/2010 20:43, Tom Igoe wrote:

> The code below fails under certain conditions. Specifically, when red hits about 0x33, it freezes up. A typical last reading before freeze is #33fffc. More interesting is what happens when red's about 0x11, see the results below. Definitely looks like it's reading memory it shouldn't beyond the String length. Furthermore, when it freezes, a simple reset won't fix it. It needs to be re-uploaded to get started again.

I doubt it's a leak. You are generating 4 String instances each time
through that loop which will fragment the malloc arena. And once malloc
fails you'll end up with a String object with an invalid internal state
that operator += will blithely use - and once you strcat the first null
pointer, all bets are off.

I can't express how bad an idea this String class is. I've already
tried and been ignored, and you are seeing exactly the sorts of problems
that your users will see. If the Arduino developers can't figure out
what's wrong, what chance to your target audience have?

--
Alan Burlison
--

_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

I think you're right, but I also don't think that telling people to
use the standard C library functions is a solution either. Trying out
this String class seems like one way of pushing forwards towards a
compromise that provides both an easy-to-use API and a robust
implementation. For example, perhaps the buffer sizes need to be
specified or provided when the String is instantiated. Or we need an
alternative implementation that does more reliable memory allocation
(since we're not assuming that the internal buffer is continuous).
Other ideas welcome.

On Wed, Dec 15, 2010 at 9:07 AM, Alan Burlison <> wrote:
> On 14/12/2010 20:43, Tom Igoe wrote:
>
>> �The code below fails under certain conditions. Specifically, when red
>> hits about 0x33, it freezes up. �A typical last reading before freeze is
>> #33fffc. �More interesting is what happens when red's about 0x11, see the
>> results below. Definitely looks like it's reading memory it shouldn't beyond
>> the String length. Furthermore, when it freezes, a simple reset won't fix
>> it. �It needs to be re-uploaded to get started again.
>
> I doubt it's a leak. �You are generating 4 String instances each time
> through that loop which will fragment the malloc arena. �And once malloc
> fails you'll end up with a String object with an invalid internal state that
> operator += will blithely use - and once you strcat the first null pointer,
> all bets are off.
>
> I can't express how bad an idea this String class is. �I've already tried
> and been ignored, and you are seeing exactly the sorts of problems that your
> users will see. �If the Arduino developers can't figure out what's wrong,
> what chance to your target audience have?
>
> --
> Alan Burlison
> --
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

Dave speaks for me in response to Alan's comments.

I wonder, would Alexander's library for memory allocation from a few days ago be useful in handling String better?

Rick, I'm going to be writing a lot of stuff with String over the semester break too, so I'm happy to give you more test cases that we can look at. With things that are not easy to fix, I can work on examples that use it in a safer way, and describe some of the pitfalls. I'll try to break stuff earlier so others don't have to.

t.


On Dec 15, 2010, at 11:25 AM, Rick Anderson wrote:

> Buried somewhere in my argument is that the String Library is not
> passing not passing memory related tests, and this is causing normal
> tests to fail unexpectedly as well. Which makes it tough to test
> thoroughly. But ultimately it is failing QA.
>
> Tom is experiencing that trouble first hand in his example. Anyone,
> trying to use the String library will likely have the same trouble.
>
> I'll have some time coming up where I can create many more tests, and
> help others develop tests for the library. What is the feasibility
> that the library can be fixed?
>
> I know there is a lot benefit to the String Library and we're trying
> to use it every where, and integrate it into many places. We would be
> bringing the quality of the product down, if this Library is
> integrated as is.
>
>
> --Rick
>
> On Wed, Dec 15, 2010 at 9:07 AM, Alan Burlison <> wrote:
>> On 14/12/2010 20:43, Tom Igoe wrote:
>>
>>> The code below fails under certain conditions. Specifically, when red
>>> hits about 0x33, it freezes up. A typical last reading before freeze is
>>> #33fffc. More interesting is what happens when red's about 0x11, see the
>>> results below. Definitely looks like it's reading memory it shouldn't beyond
>>> the String length. Furthermore, when it freezes, a simple reset won't fix
>>> it. It needs to be re-uploaded to get started again.
>>
>> I doubt it's a leak. You are generating 4 String instances each time
>> through that loop which will fragment the malloc arena. And once malloc
>> fails you'll end up with a String object with an invalid internal state that
>> operator += will blithely use - and once you strcat the first null pointer,
>> all bets are off.
>>
>> I can't express how bad an idea this String class is. I've already tried
>> and been ignored, and you are seeing exactly the sorts of problems that your
>> users will see. If the Arduino developers can't figure out what's wrong,
>> what chance to your target audience have?
>>
>> --
>> Alan Burlison
>> --
>>
>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.