This is a proposal for enhancements to Stream to support the finding and extracting numeric and string data. The parsing is done character by character so no intermediate static or dynamic buffer is required for the incoming data.



The code is based on the TextFinder library (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with help from Philip Lindsay) for the Arduino Cookbook.



The attached zip file contains 1.0 core Stream source code that implements the API described below. Examples showing use with Serial and Ethernet are also included. Please feel free to suggest improvements to the API, implementation or examples.



Here are the currently supported methods:



setTimeout()

Description: Sets the maximum number of seconds to wait for a character from the stream. The default value is 5 seconds. The timeout value is used by all of the other parsing methods and could be useful for other stream functions.

Syntax: setTimeout( timeout)

Parameters: timout: the maximum number of seconds to wait.

Returns: nothing

Example:

setTimeout(10); // set timeout to ten seconds



find()

Description: Reads data from the stream until the target string is found

Syntax: find(targetStr)

Parameters: targetStr is the null terminated C character string to search for

Returns: true if target string is found. All characters up to and including the last character in targetStr will be flushed. Returns false if the targetStr is not found within the timeout period.

Example:

if( client.find("GET /") ) { // search for 'GET'

// here if the string "GET /" is found in the stream

else

// here if find method timed out (stream buffer is empty)



findUntil()

Description: As find but search ends if the terminate string is found. Useful to stop searches at the end of a line or at a string token.

Syntax: findUntil(targetStr, terminateStr)

Parameters: targetStr is the null terminated C string to search for, terminateStr is the null terminated C string that will terminate the search if found in the stream.

Returns: true if targetStr is found, returns false if timeout or terminateStr is encountered.

Example:

// search to the end of line for “pin”

if(client.findUntil("pin", "\n\r"))

// here if “pin” is found in the current line

else

// stream buffer is at the beginning of the next line



Note that a binary variants of find and findUntil could be implemented if necessary for searching for binary data. Parameters would be a pointer to a byte array and the length of the target array. This would be functionally equivalent to the above but would handle null characters in the target and terminate strings.





getValue()

Description: Returns the first valid (long) integer value from the current position. Initial characters that are not ASCII digits (or a minus sign) are ignored. The value is terminated by the first character that is not a digit following one or more valid digits.

Syntax: getvalue()

Parameters: none

Returns: long integer value or 0 if timeout.

Example:

// this fragment will extract and print three comma separated integer values

int value;

for(int Index = 0; Index < 3; Index ++)

{

value = Serial.getValue(); // get a numeric value

Serial.println(value);

}



getValue(skipChar)

Description: As above but the given skipChar is ignored. This allows format characters (typically commas) in values to be ignored

Syntax: getvalue(skipChar)

Parameters: skipChar: a character to ignore

Returns: long integer value or 0 if timeout.

Example:

// send $12,345 terminated with a newline

int value;

value = Serial.getValue(','); // ignore commas

Serial.println(value); // prints 12345





getFloat()

Description: float version of getValue.

Syntax: getFloat()

Parameters: none

Returns: : floating point value or 0 if timeout.

Example:

// send $12.34 terminated with a newline

float value;

value = Serial.getFloat();

Serial.println(value); // prints 12.34





getFloat(skipChar)

Description: As above but the given skipChar is ignored. This allows format characters (typically commas) in values to be ignored

Syntax: getFloat(skipChar)

Parameters: skipChar: a character to ignore

Returns: floating point value or 0 if timeout.

Example:

// send $12,345.67 terminated with a newline

float value;

value = Serial.getFloat();

Serial.println(value); // prints 12345.67



getString()

Description: Finds a string between pre and post delimiters and stores into a given buffer. The string will be truncated to fit the buffer length. The string is terminated by the first character following the preString that matches the first character of the terminator.

Syntax: getString(preString, postString, buffer, length)

Parameters: preString: a string that precedes the desired target string, following characters this are appended to the given buffer until the first character of postString is encountered, length: the size of buffer.

Returns: Returns the number of characters placed in the buffer (0 means no valid data found)

Example:

char buffer[8]; // must be big enough to hold target string and terminating null

// get a string between "<" and ">"

if(client.getString( "<", ">", buffer, sizeof(buffer) ))

{

Serial.print("found ");

Serial.println(buffer);

}

This will print ‘target’ if the stream contains “”

Characters before the ‘<’ are discarded. The next character read from the stream will be the character following ‘>’



-----

This functionality has worked quite well for me but feel free to suggest improvements to the API, implementation or documentation.

You can run the examples in the zip by copying Stream.h and Stream.cpp to the cores > arduino directory of the 1.0 beta.



Michael Margolis

David, thanks for your comments.

> Would you want to be able to specify more than one skip character?
> Do we need some way to distinguish between a timeout and a real value of
> 0?
Good questions, I am interested to hear what the people here think will be
most useful.
These things are easy to add but I would like to avoid bloating the API with
features that are not likely to be used.

> For getFloat(), do we need a way to specify the character that
> separates the whole and fractional part of the number (e.g. supporting
> European formatting like 1.234,56)?
This could be added by specifying the decimal point character to a user
supplied character such as a comma (default is '.')
If locale specific configuration was needed in elsewhere in Arduino
functionality I would suggest having a locale.h file that contains defines
for things like this. But a Stream method, perhaps named
setDecimalPointChar() could be added if this functionality was only needed
in this API.

>I'd suggest using getInt() and getFloat() ...
getValue seemed less techy but I take the point about consistency.

I would be interested to hear others views on these and anything else
regarding this API.

Michael


-----Original Message-----
From: David A. Mellis
Sent: Tuesday, May 31, 2011 2:30 PM
To: Michael Margolis
Cc:
Subject: Re: [Developers] Adding parsing to Stream

Thanks for the proposal!

Does anyone have any suggestions for the API or specific use cases we
should keep in mind?

In particular, I'm curious about the precise semantics for the
getValue() functions. Would you want to be able to specify more than
one skip character? Do we need some way to distinguish between a
timeout and a real value of 0?

For getFloat(), do we need a way to specify the character that
separates the whole and fractional part of the number (e.g. supporting
European formatting like 1.234,56)?

I'd suggest using getInt() and getFloat() for consistency with the
names of the data types, even if we actually return longs and doubles.
The word "value" seems very generic.

David

On Tue, May 31, 2011 at 7:27 AM, Michael Margolis
<> wrote:
> This is a proposal for enhancements to Stream to support the finding and
> extracting numeric and string data. The parsing is done character by
> character so no intermediate static or dynamic buffer is required for the
> incoming data.
>
>
>
> The code is based on the TextFinder library
> (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with
> help
> from Philip Lindsay) for the Arduino Cookbook.
>
>
>
> The attached zip file contains 1.0 core Stream source code that implements
> the API described below. Examples showing use with Serial and Ethernet are
> also included. Please feel free to suggest improvements to the API,
> implementation or examples.
>
>
>
> Here are the currently supported methods:
>
>
>
> setTimeout()
>
> Description: Sets the maximum number of seconds to wait for a character
> from the stream. The default value is 5 seconds. The timeout value is used
> by all of the other parsing methods and could be useful for other stream
> functions.
>
> Syntax: setTimeout( timeout)
>
> Parameters: timout: the maximum number of seconds to wait.
>
> Returns: nothing
>
> Example:
>
> setTimeout(10); // set timeout to ten seconds
>
>
>
> find()
>
> Description: Reads data from the stream until the target string is found
>
> Syntax: find(targetStr)
>
> Parameters: targetStr is the null terminated C character string to search
> for
>
> Returns: true if target string is found. All characters up to and
> including
> the last character in targetStr will be flushed. Returns false if the
> targetStr is not found within the timeout period.
>
> Example:
>
> if( client.find("GET /") ) { // search for 'GET'
>
> // here if the string "GET /" is found in the stream
>
> else
>
> // here if find method timed out (stream buffer is empty)
>
>
>
> findUntil()
>
> Description: As find but search ends if the terminate string is found.
> Useful to stop searches at the end of a line or at a string token.
>
> Syntax: findUntil(targetStr, terminateStr)
>
> Parameters: targetStr is the null terminated C string to search for,
> terminateStr is the null terminated C string that will terminate the
> search
> if found in the stream.
>
> Returns: true if targetStr is found, returns false if timeout or
> terminateStr is encountered.
>
> Example:
>
> // search to the end of line for �pin�
>
> if(client.findUntil("pin", "\n\r"))
>
> // here if �pin� is found in the current line
>
> else
>
> // stream buffer is at the beginning of the next line
>
>
>
> Note that a binary variants of find and findUntil could be implemented if
> necessary for searching for binary data. Parameters would be a pointer to
> a
> byte array and the length of the target array. This would be functionally
> equivalent to the above but would handle null characters in the target and
> terminate strings.
>
>
>
>
>
> getValue()
>
> Description: Returns the first valid (long) integer value from the
> current
> position. Initial characters that are not ASCII digits (or a minus sign)
> are
> ignored. The value is terminated by the first character that is not a
> digit
> following one or more valid digits.
>
> Syntax: getvalue()
>
> Parameters: none
>
> Returns: long integer value or 0 if timeout.
>
> Example:
>
> // this fragment will extract and print three comma separated integer
> values
>
> int value;
>
> for(int Index = 0; Index < 3; Index ++)
>
> {
>
> value = Serial.getValue(); // get a numeric value
>
> Serial.println(value);
>
> }
>
>
>
> getValue(skipChar)
>
> Description: As above but the given skipChar is ignored. This allows
> format
> characters (typically commas) in values to be ignored
>
> Syntax: getvalue(skipChar)
>
> Parameters: skipChar: a character to ignore
>
> Returns: long integer value or 0 if timeout.
>
> Example:
>
> // send $12,345 terminated with a newline
>
> int value;
>
> value = Serial.getValue(','); // ignore commas
>
> Serial.println(value); // prints 12345
>
>
>
>
>
> getFloat()
>
> Description: float version of getValue.
>
> Syntax: getFloat()
>
> Parameters: none
>
> Returns: : floating point value or 0 if timeout.
>
> Example:
>
> // send $12.34 terminated with a newline
>
> float value;
>
> value = Serial.getFloat();
>
> Serial.println(value); // prints 12.34
>
>
>
>
>
> getFloat(skipChar)
>
> Description: As above but the given skipChar is ignored. This allows
> format
> characters (typically commas) in values to be ignored
>
> Syntax: getFloat(skipChar)
>
> Parameters: skipChar: a character to ignore
>
> Returns: floating point value or 0 if timeout.
>
> Example:
>
> // send $12,345.67 terminated with a newline
>
> float value;
>
> value = Serial.getFloat();
>
> Serial.println(value); // prints 12345.67
>
>
>
> getString()
>
> Description: Finds a string between pre and post delimiters and stores
> into
> a given buffer. The string will be truncated to fit the buffer length. The
> string is terminated by the first character following the preString that
> matches the first character of the terminator.
>
> Syntax: getString(preString, postString, buffer, length)
>
> Parameters: preString: a string that precedes the desired target string,
> following characters this are appended to the given buffer until the first
> character of postString is encountered, length: the size of buffer.
>
> Returns: Returns the number of characters placed in the buffer (0 means
> no
> valid data found)
>
> Example:
>
> char buffer[8]; // must be big enough to hold target string and
> terminating null
>
> // get a string between "<" and ">"
>
> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>
> {
>
> Serial.print("found ");
>
> Serial.println(buffer);
>
> }
>
> This will print �target� if the stream contains ��
>
> Characters before the �<� are discarded. The next character read from the
> stream will be the character following �>�
>
>
>
> -----
>
> This functionality has worked quite well for me but feel free to suggest
> improvements to the API, implementation or documentation.
>
> You can run the examples in the zip by copying Stream.h and Stream.cpp to
> the cores > arduino directory of the 1.0 beta.
>
>
>
> Michael Margolis
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

Thus work looks great! One use case I've been thinking a lot about is HTTP response parsing. I'm seeing more and more users start playing with the Ethernet library and nearly all immediately run into a wall when trying to parse the incoming requests and responses into useful header data (i.e. response code and, secondarily, other header values) and a body whose payload they can also parse to extract their actual data.

I've been thinking about putting together a library that would wrap the Ethernet library a bit to provide a much simpler API that didn't ask the user to do any parsing or stream handling except possibly parsing their own custom format in the body.

This parsing API seems like a great toolkit to build that library with.

-- Greg

On May 31, 2011, at 10:32 AM, "Michael Margolis" <> wrote:

> David, thanks for your comments.
>
>> Would you want to be able to specify more than one skip character?
>> Do we need some way to distinguish between a timeout and a real value of 0?
> Good questions, I am interested to hear what the people here think will be most useful.
> These things are easy to add but I would like to avoid bloating the API with features that are not likely to be used.
>
>> For getFloat(), do we need a way to specify the character that
>> separates the whole and fractional part of the number (e.g. supporting
>> European formatting like 1.234,56)?
> This could be added by specifying the decimal point character to a user supplied character such as a comma (default is '.')
> If locale specific configuration was needed in elsewhere in Arduino functionality I would suggest having a locale.h file that contains defines for things like this. But a Stream method, perhaps named setDecimalPointChar() could be added if this functionality was only needed in this API.
>
>> I'd suggest using getInt() and getFloat() ...
> getValue seemed less techy but I take the point about consistency.
>
> I would be interested to hear others views on these and anything else regarding this API.
>
> Michael
>
>
> -----Original Message----- From: David A. Mellis
> Sent: Tuesday, May 31, 2011 2:30 PM
> To: Michael Margolis
> Cc:
> Subject: Re: [Developers] Adding parsing to Stream
>
> Thanks for the proposal!
>
> Does anyone have any suggestions for the API or specific use cases we
> should keep in mind?
>
> In particular, I'm curious about the precise semantics for the
> getValue() functions. Would you want to be able to specify more than
> one skip character? Do we need some way to distinguish between a
> timeout and a real value of 0?
>
> For getFloat(), do we need a way to specify the character that
> separates the whole and fractional part of the number (e.g. supporting
> European formatting like 1.234,56)?
>
> I'd suggest using getInt() and getFloat() for consistency with the
> names of the data types, even if we actually return longs and doubles.
> The word "value" seems very generic.
>
> David
>
> On Tue, May 31, 2011 at 7:27 AM, Michael Margolis
> <> wrote:
>> This is a proposal for enhancements to Stream to support the finding and
>> extracting numeric and string data. The parsing is done character by
>> character so no intermediate static or dynamic buffer is required for the
>> incoming data.
>>
>>
>>
>> The code is based on the TextFinder library
>> (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with help
>> from Philip Lindsay) for the Arduino Cookbook.
>>
>>
>>
>> The attached zip file contains 1.0 core Stream source code that implements
>> the API described below. Examples showing use with Serial and Ethernet are
>> also included. Please feel free to suggest improvements to the API,
>> implementation or examples.
>>
>>
>>
>> Here are the currently supported methods:
>>
>>
>>
>> setTimeout()
>>
>> Description: Sets the maximum number of seconds to wait for a character
>> from the stream. The default value is 5 seconds. The timeout value is used
>> by all of the other parsing methods and could be useful for other stream
>> functions.
>>
>> Syntax: setTimeout( timeout)
>>
>> Parameters: timout: the maximum number of seconds to wait.
>>
>> Returns: nothing
>>
>> Example:
>>
>> setTimeout(10); // set timeout to ten seconds
>>
>>
>>
>> find()
>>
>> Description: Reads data from the stream until the target string is found
>>
>> Syntax: find(targetStr)
>>
>> Parameters: targetStr is the null terminated C character string to search
>> for
>>
>> Returns: true if target string is found. All characters up to and including
>> the last character in targetStr will be flushed. Returns false if the
>> targetStr is not found within the timeout period.
>>
>> Example:
>>
>> if( client.find("GET /") ) { // search for 'GET'
>>
>> // here if the string "GET /" is found in the stream
>>
>> else
>>
>> // here if find method timed out (stream buffer is empty)
>>
>>
>>
>> findUntil()
>>
>> Description: As find but search ends if the terminate string is found.
>> Useful to stop searches at the end of a line or at a string token.
>>
>> Syntax: findUntil(targetStr, terminateStr)
>>
>> Parameters: targetStr is the null terminated C string to search for,
>> terminateStr is the null terminated C string that will terminate the search
>> if found in the stream.
>>
>> Returns: true if targetStr is found, returns false if timeout or
>> terminateStr is encountered.
>>
>> Example:
>>
>> // search to the end of line for “pin”
>>
>> if(client.findUntil("pin", "\n\r"))
>>
>> // here if “pin” is found in the current line
>>
>> else
>>
>> // stream buffer is at the beginning of the next line
>>
>>
>>
>> Note that a binary variants of find and findUntil could be implemented if
>> necessary for searching for binary data. Parameters would be a pointer to a
>> byte array and the length of the target array. This would be functionally
>> equivalent to the above but would handle null characters in the target and
>> terminate strings.
>>
>>
>>
>>
>>
>> getValue()
>>
>> Description: Returns the first valid (long) integer value from the current
>> position. Initial characters that are not ASCII digits (or a minus sign) are
>> ignored. The value is terminated by the first character that is not a digit
>> following one or more valid digits.
>>
>> Syntax: getvalue()
>>
>> Parameters: none
>>
>> Returns: long integer value or 0 if timeout.
>>
>> Example:
>>
>> // this fragment will extract and print three comma separated integer
>> values
>>
>> int value;
>>
>> for(int Index = 0; Index < 3; Index ++)
>>
>> {
>>
>> value = Serial.getValue(); // get a numeric value
>>
>> Serial.println(value);
>>
>> }
>>
>>
>>
>> getValue(skipChar)
>>
>> Description: As above but the given skipChar is ignored. This allows format
>> characters (typically commas) in values to be ignored
>>
>> Syntax: getvalue(skipChar)
>>
>> Parameters: skipChar: a character to ignore
>>
>> Returns: long integer value or 0 if timeout.
>>
>> Example:
>>
>> // send $12,345 terminated with a newline
>>
>> int value;
>>
>> value = Serial.getValue(','); // ignore commas
>>
>> Serial.println(value); // prints 12345
>>
>>
>>
>>
>>
>> getFloat()
>>
>> Description: float version of getValue.
>>
>> Syntax: getFloat()
>>
>> Parameters: none
>>
>> Returns: : floating point value or 0 if timeout.
>>
>> Example:
>>
>> // send $12.34 terminated with a newline
>>
>> float value;
>>
>> value = Serial.getFloat();
>>
>> Serial.println(value); // prints 12.34
>>
>>
>>
>>
>>
>> getFloat(skipChar)
>>
>> Description: As above but the given skipChar is ignored. This allows format
>> characters (typically commas) in values to be ignored
>>
>> Syntax: getFloat(skipChar)
>>
>> Parameters: skipChar: a character to ignore
>>
>> Returns: floating point value or 0 if timeout.
>>
>> Example:
>>
>> // send $12,345.67 terminated with a newline
>>
>> float value;
>>
>> value = Serial.getFloat();
>>
>> Serial.println(value); // prints 12345.67
>>
>>
>>
>> getString()
>>
>> Description: Finds a string between pre and post delimiters and stores into
>> a given buffer. The string will be truncated to fit the buffer length. The
>> string is terminated by the first character following the preString that
>> matches the first character of the terminator.
>>
>> Syntax: getString(preString, postString, buffer, length)
>>
>> Parameters: preString: a string that precedes the desired target string,
>> following characters this are appended to the given buffer until the first
>> character of postString is encountered, length: the size of buffer.
>>
>> Returns: Returns the number of characters placed in the buffer (0 means no
>> valid data found)
>>
>> Example:
>>
>> char buffer[8]; // must be big enough to hold target string and
>> terminating null
>>
>> // get a string between "<" and ">"
>>
>> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>>
>> {
>>
>> Serial.print("found ");
>>
>> Serial.println(buffer);
>>
>> }
>>
>> This will print ‘target’ if the stream contains “”
>>
>> Characters before the ‘<’ are discarded. The next character read from the
>> stream will be the character following ‘>’
>>
>>
>>
>> -----
>>
>> This functionality has worked quite well for me but feel free to suggest
>> improvements to the API, implementation or documentation.
>>
>> You can run the examples in the zip by copying Stream.h and Stream.cpp to
>> the cores > arduino directory of the 1.0 beta.
>>
>>
>>
>> Michael Margolis
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

i have been using TextFinder in the rewrite of Making Things talk and it's very handy. I'd love to see it in the core. I'd love to see find() and findUntil() use peek() perhaps, so that you can do multiple finds() on a stream. I've had some cases where I needed to switch from find() to a character based search. For example, if I'm looking for

This is what I want

I have to find() for the then read() until I see a <. Maybe I'm missing something, but it'd be nice to just slurp everything between and

t.





On May 31, 2011, at 10:32 AM, Michael Margolis wrote:

> David, thanks for your comments.
>
>> Would you want to be able to specify more than one skip character?
>> Do we need some way to distinguish between a timeout and a real value of 0?
> Good questions, I am interested to hear what the people here think will be most useful.
> These things are easy to add but I would like to avoid bloating the API with features that are not likely to be used.
>
>> For getFloat(), do we need a way to specify the character that
>> separates the whole and fractional part of the number (e.g. supporting
>> European formatting like 1.234,56)?
> This could be added by specifying the decimal point character to a user supplied character such as a comma (default is '.')
> If locale specific configuration was needed in elsewhere in Arduino functionality I would suggest having a locale.h file that contains defines for things like this. But a Stream method, perhaps named setDecimalPointChar() could be added if this functionality was only needed in this API.
>
>> I'd suggest using getInt() and getFloat() ...
> getValue seemed less techy but I take the point about consistency.
>
> I would be interested to hear others views on these and anything else regarding this API.
>
> Michael
>
>
> -----Original Message----- From: David A. Mellis
> Sent: Tuesday, May 31, 2011 2:30 PM
> To: Michael Margolis
> Cc:
> Subject: Re: [Developers] Adding parsing to Stream
>
> Thanks for the proposal!
>
> Does anyone have any suggestions for the API or specific use cases we
> should keep in mind?
>
> In particular, I'm curious about the precise semantics for the
> getValue() functions. Would you want to be able to specify more than
> one skip character? Do we need some way to distinguish between a
> timeout and a real value of 0?
>
> For getFloat(), do we need a way to specify the character that
> separates the whole and fractional part of the number (e.g. supporting
> European formatting like 1.234,56)?
>
> I'd suggest using getInt() and getFloat() for consistency with the
> names of the data types, even if we actually return longs and doubles.
> The word "value" seems very generic.
>
> David
>
> On Tue, May 31, 2011 at 7:27 AM, Michael Margolis
> <> wrote:
>> This is a proposal for enhancements to Stream to support the finding and
>> extracting numeric and string data. The parsing is done character by
>> character so no intermediate static or dynamic buffer is required for the
>> incoming data.
>>
>>
>>
>> The code is based on the TextFinder library
>> (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with help
>> from Philip Lindsay) for the Arduino Cookbook.
>>
>>
>>
>> The attached zip file contains 1.0 core Stream source code that implements
>> the API described below. Examples showing use with Serial and Ethernet are
>> also included. Please feel free to suggest improvements to the API,
>> implementation or examples.
>>
>>
>>
>> Here are the currently supported methods:
>>
>>
>>
>> setTimeout()
>>
>> Description: Sets the maximum number of seconds to wait for a character
>> from the stream. The default value is 5 seconds. The timeout value is used
>> by all of the other parsing methods and could be useful for other stream
>> functions.
>>
>> Syntax: setTimeout( timeout)
>>
>> Parameters: timout: the maximum number of seconds to wait.
>>
>> Returns: nothing
>>
>> Example:
>>
>> setTimeout(10); // set timeout to ten seconds
>>
>>
>>
>> find()
>>
>> Description: Reads data from the stream until the target string is found
>>
>> Syntax: find(targetStr)
>>
>> Parameters: targetStr is the null terminated C character string to search
>> for
>>
>> Returns: true if target string is found. All characters up to and including
>> the last character in targetStr will be flushed. Returns false if the
>> targetStr is not found within the timeout period.
>>
>> Example:
>>
>> if( client.find("GET /") ) { // search for 'GET'
>>
>> // here if the string "GET /" is found in the stream
>>
>> else
>>
>> // here if find method timed out (stream buffer is empty)
>>
>>
>>
>> findUntil()
>>
>> Description: As find but search ends if the terminate string is found.
>> Useful to stop searches at the end of a line or at a string token.
>>
>> Syntax: findUntil(targetStr, terminateStr)
>>
>> Parameters: targetStr is the null terminated C string to search for,
>> terminateStr is the null terminated C string that will terminate the search
>> if found in the stream.
>>
>> Returns: true if targetStr is found, returns false if timeout or
>> terminateStr is encountered.
>>
>> Example:
>>
>> // search to the end of line for �pin�
>>
>> if(client.findUntil("pin", "\n\r"))
>>
>> // here if �pin� is found in the current line
>>
>> else
>>
>> // stream buffer is at the beginning of the next line
>>
>>
>>
>> Note that a binary variants of find and findUntil could be implemented if
>> necessary for searching for binary data. Parameters would be a pointer to a
>> byte array and the length of the target array. This would be functionally
>> equivalent to the above but would handle null characters in the target and
>> terminate strings.
>>
>>
>>
>>
>>
>> getValue()
>>
>> Description: Returns the first valid (long) integer value from the current
>> position. Initial characters that are not ASCII digits (or a minus sign) are
>> ignored. The value is terminated by the first character that is not a digit
>> following one or more valid digits.
>>
>> Syntax: getvalue()
>>
>> Parameters: none
>>
>> Returns: long integer value or 0 if timeout.
>>
>> Example:
>>
>> // this fragment will extract and print three comma separated integer
>> values
>>
>> int value;
>>
>> for(int Index = 0; Index < 3; Index ++)
>>
>> {
>>
>> value = Serial.getValue(); // get a numeric value
>>
>> Serial.println(value);
>>
>> }
>>
>>
>>
>> getValue(skipChar)
>>
>> Description: As above but the given skipChar is ignored. This allows format
>> characters (typically commas) in values to be ignored
>>
>> Syntax: getvalue(skipChar)
>>
>> Parameters: skipChar: a character to ignore
>>
>> Returns: long integer value or 0 if timeout.
>>
>> Example:
>>
>> // send $12,345 terminated with a newline
>>
>> int value;
>>
>> value = Serial.getValue(','); // ignore commas
>>
>> Serial.println(value); // prints 12345
>>
>>
>>
>>
>>
>> getFloat()
>>
>> Description: float version of getValue.
>>
>> Syntax: getFloat()
>>
>> Parameters: none
>>
>> Returns: : floating point value or 0 if timeout.
>>
>> Example:
>>
>> // send $12.34 terminated with a newline
>>
>> float value;
>>
>> value = Serial.getFloat();
>>
>> Serial.println(value); // prints 12.34
>>
>>
>>
>>
>>
>> getFloat(skipChar)
>>
>> Description: As above but the given skipChar is ignored. This allows format
>> characters (typically commas) in values to be ignored
>>
>> Syntax: getFloat(skipChar)
>>
>> Parameters: skipChar: a character to ignore
>>
>> Returns: floating point value or 0 if timeout.
>>
>> Example:
>>
>> // send $12,345.67 terminated with a newline
>>
>> float value;
>>
>> value = Serial.getFloat();
>>
>> Serial.println(value); // prints 12345.67
>>
>>
>>
>> getString()
>>
>> Description: Finds a string between pre and post delimiters and stores into
>> a given buffer. The string will be truncated to fit the buffer length. The
>> string is terminated by the first character following the preString that
>> matches the first character of the terminator.
>>
>> Syntax: getString(preString, postString, buffer, length)
>>
>> Parameters: preString: a string that precedes the desired target string,
>> following characters this are appended to the given buffer until the first
>> character of postString is encountered, length: the size of buffer.
>>
>> Returns: Returns the number of characters placed in the buffer (0 means no
>> valid data found)
>>
>> Example:
>>
>> char buffer[8]; // must be big enough to hold target string and
>> terminating null
>>
>> // get a string between "<" and ">"
>>
>> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>>
>> {
>>
>> Serial.print("found ");
>>
>> Serial.println(buffer);
>>
>> }
>>
>> This will print �target� if the stream contains ��
>>
>> Characters before the �<� are discarded. The next character read from the
>> stream will be the character following �>�
>>
>>
>>
>> -----
>>
>> This functionality has worked quite well for me but feel free to suggest
>> improvements to the API, implementation or documentation.
>>
>> You can run the examples in the zip by copying Stream.h and Stream.cpp to
>> the cores > arduino directory of the 1.0 beta.
>>
>>
>>
>> Michael Margolis
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

Tom, thanks for your comments.

The challenge with using peek is handling cases where the number of
characters in the document is greater than the buffer size. For example,
the google finance example I use in the Arduino Cookbook extracts a stock
price that is typically 50k characters beyond the document head. If useful
I could add a peek(target) and peekUntil(targetStr, terminateStr) that
returned the location of the given target in the buffer or -1 if not found
in the buffer. But I wonder if this functionality would be useful if the
data of interest exceeded the stream buffer size.

Michael

-----Original Message-----
From: Tom Igoe
Sent: Tuesday, May 31, 2011 4:53 PM
To: Michael Margolis
Cc: David A. Mellis ;
Subject: Re: [Developers] Adding parsing to Stream

i have been using TextFinder in the rewrite of Making Things talk and it's
very handy. I'd love to see it in the core. I'd love to see find() and
findUntil() use peek() perhaps, so that you can do multiple finds() on a
stream. I've had some cases where I needed to switch from find() to a
character based search. For example, if I'm looking for

This is what I want

I have to find() for the then read() until I see a <. Maybe I'm
missing something, but it'd be nice to just slurp everything between
and

t.





On May 31, 2011, at 10:32 AM, Michael Margolis wrote:

> David, thanks for your comments.
>
>> Would you want to be able to specify more than one skip character?
>> Do we need some way to distinguish between a timeout and a real value of
>> 0?
> Good questions, I am interested to hear what the people here think will be
> most useful.
> These things are easy to add but I would like to avoid bloating the API
> with features that are not likely to be used.
>
>> For getFloat(), do we need a way to specify the character that
>> separates the whole and fractional part of the number (e.g. supporting
>> European formatting like 1.234,56)?
> This could be added by specifying the decimal point character to a user
> supplied character such as a comma (default is '.')
> If locale specific configuration was needed in elsewhere in Arduino
> functionality I would suggest having a locale.h file that contains defines
> for things like this. But a Stream method, perhaps named
> setDecimalPointChar() could be added if this functionality was only
> needed in this API.
>
>> I'd suggest using getInt() and getFloat() ...
> getValue seemed less techy but I take the point about consistency.
>
> I would be interested to hear others views on these and anything else
> regarding this API.
>
> Michael
>
>
> -----Original Message----- From: David A. Mellis
> Sent: Tuesday, May 31, 2011 2:30 PM
> To: Michael Margolis
> Cc:
> Subject: Re: [Developers] Adding parsing to Stream
>
> Thanks for the proposal!
>
> Does anyone have any suggestions for the API or specific use cases we
> should keep in mind?
>
> In particular, I'm curious about the precise semantics for the
> getValue() functions. Would you want to be able to specify more than
> one skip character? Do we need some way to distinguish between a
> timeout and a real value of 0?
>
> For getFloat(), do we need a way to specify the character that
> separates the whole and fractional part of the number (e.g. supporting
> European formatting like 1.234,56)?
>
> I'd suggest using getInt() and getFloat() for consistency with the
> names of the data types, even if we actually return longs and doubles.
> The word "value" seems very generic.
>
> David
>
> On Tue, May 31, 2011 at 7:27 AM, Michael Margolis
> <> wrote:
>> This is a proposal for enhancements to Stream to support the finding and
>> extracting numeric and string data. The parsing is done character by
>> character so no intermediate static or dynamic buffer is required for the
>> incoming data.
>>
>>
>>
>> The code is based on the TextFinder library
>> (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with
>> help
>> from Philip Lindsay) for the Arduino Cookbook.
>>
>>
>>
>> The attached zip file contains 1.0 core Stream source code that
>> implements
>> the API described below. Examples showing use with Serial and Ethernet
>> are
>> also included. Please feel free to suggest improvements to the API,
>> implementation or examples.
>>
>>
>>
>> Here are the currently supported methods:
>>
>>
>>
>> setTimeout()
>>
>> Description: Sets the maximum number of seconds to wait for a character
>> from the stream. The default value is 5 seconds. The timeout value is
>> used
>> by all of the other parsing methods and could be useful for other stream
>> functions.
>>
>> Syntax: setTimeout( timeout)
>>
>> Parameters: timout: the maximum number of seconds to wait.
>>
>> Returns: nothing
>>
>> Example:
>>
>> setTimeout(10); // set timeout to ten seconds
>>
>>
>>
>> find()
>>
>> Description: Reads data from the stream until the target string is found
>>
>> Syntax: find(targetStr)
>>
>> Parameters: targetStr is the null terminated C character string to search
>> for
>>
>> Returns: true if target string is found. All characters up to and
>> including
>> the last character in targetStr will be flushed. Returns false if the
>> targetStr is not found within the timeout period.
>>
>> Example:
>>
>> if( client.find("GET /") ) { // search for 'GET'
>>
>> // here if the string "GET /" is found in the stream
>>
>> else
>>
>> // here if find method timed out (stream buffer is empty)
>>
>>
>>
>> findUntil()
>>
>> Description: As find but search ends if the terminate string is found.
>> Useful to stop searches at the end of a line or at a string token.
>>
>> Syntax: findUntil(targetStr, terminateStr)
>>
>> Parameters: targetStr is the null terminated C string to search for,
>> terminateStr is the null terminated C string that will terminate the
>> search
>> if found in the stream.
>>
>> Returns: true if targetStr is found, returns false if timeout or
>> terminateStr is encountered.
>>
>> Example:
>>
>> // search to the end of line for �pin�
>>
>> if(client.findUntil("pin", "\n\r"))
>>
>> // here if �pin� is found in the current line
>>
>> else
>>
>> // stream buffer is at the beginning of the next line
>>
>>
>>
>> Note that a binary variants of find and findUntil could be implemented if
>> necessary for searching for binary data. Parameters would be a pointer to
>> a
>> byte array and the length of the target array. This would be functionally
>> equivalent to the above but would handle null characters in the target
>> and
>> terminate strings.
>>
>>
>>
>>
>>
>> getValue()
>>
>> Description: Returns the first valid (long) integer value from the
>> current
>> position. Initial characters that are not ASCII digits (or a minus sign)
>> are
>> ignored. The value is terminated by the first character that is not a
>> digit
>> following one or more valid digits.
>>
>> Syntax: getvalue()
>>
>> Parameters: none
>>
>> Returns: long integer value or 0 if timeout.
>>
>> Example:
>>
>> // this fragment will extract and print three comma separated integer
>> values
>>
>> int value;
>>
>> for(int Index = 0; Index < 3; Index ++)
>>
>> {
>>
>> value = Serial.getValue(); // get a numeric value
>>
>> Serial.println(value);
>>
>> }
>>
>>
>>
>> getValue(skipChar)
>>
>> Description: As above but the given skipChar is ignored. This allows
>> format
>> characters (typically commas) in values to be ignored
>>
>> Syntax: getvalue(skipChar)
>>
>> Parameters: skipChar: a character to ignore
>>
>> Returns: long integer value or 0 if timeout.
>>
>> Example:
>>
>> // send $12,345 terminated with a newline
>>
>> int value;
>>
>> value = Serial.getValue(','); // ignore commas
>>
>> Serial.println(value); // prints 12345
>>
>>
>>
>>
>>
>> getFloat()
>>
>> Description: float version of getValue.
>>
>> Syntax: getFloat()
>>
>> Parameters: none
>>
>> Returns: : floating point value or 0 if timeout.
>>
>> Example:
>>
>> // send $12.34 terminated with a newline
>>
>> float value;
>>
>> value = Serial.getFloat();
>>
>> Serial.println(value); // prints 12.34
>>
>>
>>
>>
>>
>> getFloat(skipChar)
>>
>> Description: As above but the given skipChar is ignored. This allows
>> format
>> characters (typically commas) in values to be ignored
>>
>> Syntax: getFloat(skipChar)
>>
>> Parameters: skipChar: a character to ignore
>>
>> Returns: floating point value or 0 if timeout.
>>
>> Example:
>>
>> // send $12,345.67 terminated with a newline
>>
>> float value;
>>
>> value = Serial.getFloat();
>>
>> Serial.println(value); // prints 12345.67
>>
>>
>>
>> getString()
>>
>> Description: Finds a string between pre and post delimiters and stores
>> into
>> a given buffer. The string will be truncated to fit the buffer length.
>> The
>> string is terminated by the first character following the preString that
>> matches the first character of the terminator.
>>
>> Syntax: getString(preString, postString, buffer, length)
>>
>> Parameters: preString: a string that precedes the desired target string,
>> following characters this are appended to the given buffer until the
>> first
>> character of postString is encountered, length: the size of buffer.
>>
>> Returns: Returns the number of characters placed in the buffer (0 means
>> no
>> valid data found)
>>
>> Example:
>>
>> char buffer[8]; // must be big enough to hold target string and
>> terminating null
>>
>> // get a string between "<" and ">"
>>
>> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>>
>> {
>>
>> Serial.print("found ");
>>
>> Serial.println(buffer);
>>
>> }
>>
>> This will print �target� if the stream contains ��
>>
>> Characters before the �<� are discarded. The next character read from the
>> stream will be the character following �>�
>>
>>
>>
>> -----
>>
>> This functionality has worked quite well for me but feel free to suggest
>> improvements to the API, implementation or documentation.
>>
>> You can run the examples in the zip by copying Stream.h and Stream.cpp to
>> the cores > arduino directory of the 1.0 beta.
>>
>>
>>
>> Michael Margolis
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

Yeah, I know what you mean, it's a challenge,because there's so much stuff you have to filter through to get to the part you want, much of the time. Ideally, I guess the process is something like this:

find()
appendToTempBufferUntil()
copy temp buffer to working variable or whatever

I guess you could do a saveUntil(), but that would require a dynamically allocatable temp buffer, and that's ugly. Probably better to just leave it as is and do the combo of find() then read().

t.



On May 31, 2011, at 12:56 PM, Michael Margolis wrote:

> Tom, thanks for your comments.
>
> The challenge with using peek is handling cases where the number of characters in the document is greater than the buffer size. For example, the google finance example I use in the Arduino Cookbook extracts a stock price that is typically 50k characters beyond the document head. If useful I could add a peek(target) and peekUntil(targetStr, terminateStr) that returned the location of the given target in the buffer or -1 if not found in the buffer. But I wonder if this functionality would be useful if the data of interest exceeded the stream buffer size.
>
> Michael
>
> -----Original Message----- From: Tom Igoe
> Sent: Tuesday, May 31, 2011 4:53 PM
> To: Michael Margolis
> Cc: David A. Mellis ;
> Subject: Re: [Developers] Adding parsing to Stream
>
> i have been using TextFinder in the rewrite of Making Things talk and it's very handy. I'd love to see it in the core. I'd love to see find() and findUntil() use peek() perhaps, so that you can do multiple finds() on a stream. I've had some cases where I needed to switch from find() to a character based search. For example, if I'm looking for
>
> This is what I want
>
> I have to find() for the then read() until I see a <. Maybe I'm missing something, but it'd be nice to just slurp everything between and
>
> t.
>
>
>
>
>
> On May 31, 2011, at 10:32 AM, Michael Margolis wrote:
>
>> David, thanks for your comments.
>>
>>> Would you want to be able to specify more than one skip character?
>>> Do we need some way to distinguish between a timeout and a real value of 0?
>> Good questions, I am interested to hear what the people here think will be most useful.
>> These things are easy to add but I would like to avoid bloating the API with features that are not likely to be used.
>>
>>> For getFloat(), do we need a way to specify the character that
>>> separates the whole and fractional part of the number (e.g. supporting
>>> European formatting like 1.234,56)?
>> This could be added by specifying the decimal point character to a user supplied character such as a comma (default is '.')
>> If locale specific configuration was needed in elsewhere in Arduino functionality I would suggest having a locale.h file that contains defines for things like this. But a Stream method, perhaps named setDecimalPointChar() could be added if this functionality was only needed in this API.
>>
>>> I'd suggest using getInt() and getFloat() ...
>> getValue seemed less techy but I take the point about consistency.
>>
>> I would be interested to hear others views on these and anything else regarding this API.
>>
>> Michael
>>
>>
>> -----Original Message----- From: David A. Mellis
>> Sent: Tuesday, May 31, 2011 2:30 PM
>> To: Michael Margolis
>> Cc:
>> Subject: Re: [Developers] Adding parsing to Stream
>>
>> Thanks for the proposal!
>>
>> Does anyone have any suggestions for the API or specific use cases we
>> should keep in mind?
>>
>> In particular, I'm curious about the precise semantics for the
>> getValue() functions. Would you want to be able to specify more than
>> one skip character? Do we need some way to distinguish between a
>> timeout and a real value of 0?
>>
>> For getFloat(), do we need a way to specify the character that
>> separates the whole and fractional part of the number (e.g. supporting
>> European formatting like 1.234,56)?
>>
>> I'd suggest using getInt() and getFloat() for consistency with the
>> names of the data types, even if we actually return longs and doubles.
>> The word "value" seems very generic.
>>
>> David
>>
>> On Tue, May 31, 2011 at 7:27 AM, Michael Margolis
>> <> wrote:
>>> This is a proposal for enhancements to Stream to support the finding and
>>> extracting numeric and string data. The parsing is done character by
>>> character so no intermediate static or dynamic buffer is required for the
>>> incoming data.
>>>
>>>
>>>
>>> The code is based on the TextFinder library
>>> (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with help
>>> from Philip Lindsay) for the Arduino Cookbook.
>>>
>>>
>>>
>>> The attached zip file contains 1.0 core Stream source code that implements
>>> the API described below. Examples showing use with Serial and Ethernet are
>>> also included. Please feel free to suggest improvements to the API,
>>> implementation or examples.
>>>
>>>
>>>
>>> Here are the currently supported methods:
>>>
>>>
>>>
>>> setTimeout()
>>>
>>> Description: Sets the maximum number of seconds to wait for a character
>>> from the stream. The default value is 5 seconds. The timeout value is used
>>> by all of the other parsing methods and could be useful for other stream
>>> functions.
>>>
>>> Syntax: setTimeout( timeout)
>>>
>>> Parameters: timout: the maximum number of seconds to wait.
>>>
>>> Returns: nothing
>>>
>>> Example:
>>>
>>> setTimeout(10); // set timeout to ten seconds
>>>
>>>
>>>
>>> find()
>>>
>>> Description: Reads data from the stream until the target string is found
>>>
>>> Syntax: find(targetStr)
>>>
>>> Parameters: targetStr is the null terminated C character string to search
>>> for
>>>
>>> Returns: true if target string is found. All characters up to and including
>>> the last character in targetStr will be flushed. Returns false if the
>>> targetStr is not found within the timeout period.
>>>
>>> Example:
>>>
>>> if( client.find("GET /") ) { // search for 'GET'
>>>
>>> // here if the string "GET /" is found in the stream
>>>
>>> else
>>>
>>> // here if find method timed out (stream buffer is empty)
>>>
>>>
>>>
>>> findUntil()
>>>
>>> Description: As find but search ends if the terminate string is found.
>>> Useful to stop searches at the end of a line or at a string token.
>>>
>>> Syntax: findUntil(targetStr, terminateStr)
>>>
>>> Parameters: targetStr is the null terminated C string to search for,
>>> terminateStr is the null terminated C string that will terminate the search
>>> if found in the stream.
>>>
>>> Returns: true if targetStr is found, returns false if timeout or
>>> terminateStr is encountered.
>>>
>>> Example:
>>>
>>> // search to the end of line for �pin�
>>>
>>> if(client.findUntil("pin", "\n\r"))
>>>
>>> // here if �pin� is found in the current line
>>>
>>> else
>>>
>>> // stream buffer is at the beginning of the next line
>>>
>>>
>>>
>>> Note that a binary variants of find and findUntil could be implemented if
>>> necessary for searching for binary data. Parameters would be a pointer to a
>>> byte array and the length of the target array. This would be functionally
>>> equivalent to the above but would handle null characters in the target and
>>> terminate strings.
>>>
>>>
>>>
>>>
>>>
>>> getValue()
>>>
>>> Description: Returns the first valid (long) integer value from the current
>>> position. Initial characters that are not ASCII digits (or a minus sign) are
>>> ignored. The value is terminated by the first character that is not a digit
>>> following one or more valid digits.
>>>
>>> Syntax: getvalue()
>>>
>>> Parameters: none
>>>
>>> Returns: long integer value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // this fragment will extract and print three comma separated integer
>>> values
>>>
>>> int value;
>>>
>>> for(int Index = 0; Index < 3; Index ++)
>>>
>>> {
>>>
>>> value = Serial.getValue(); // get a numeric value
>>>
>>> Serial.println(value);
>>>
>>> }
>>>
>>>
>>>
>>> getValue(skipChar)
>>>
>>> Description: As above but the given skipChar is ignored. This allows format
>>> characters (typically commas) in values to be ignored
>>>
>>> Syntax: getvalue(skipChar)
>>>
>>> Parameters: skipChar: a character to ignore
>>>
>>> Returns: long integer value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12,345 terminated with a newline
>>>
>>> int value;
>>>
>>> value = Serial.getValue(','); // ignore commas
>>>
>>> Serial.println(value); // prints 12345
>>>
>>>
>>>
>>>
>>>
>>> getFloat()
>>>
>>> Description: float version of getValue.
>>>
>>> Syntax: getFloat()
>>>
>>> Parameters: none
>>>
>>> Returns: : floating point value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12.34 terminated with a newline
>>>
>>> float value;
>>>
>>> value = Serial.getFloat();
>>>
>>> Serial.println(value); // prints 12.34
>>>
>>>
>>>
>>>
>>>
>>> getFloat(skipChar)
>>>
>>> Description: As above but the given skipChar is ignored. This allows format
>>> characters (typically commas) in values to be ignored
>>>
>>> Syntax: getFloat(skipChar)
>>>
>>> Parameters: skipChar: a character to ignore
>>>
>>> Returns: floating point value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12,345.67 terminated with a newline
>>>
>>> float value;
>>>
>>> value = Serial.getFloat();
>>>
>>> Serial.println(value); // prints 12345.67
>>>
>>>
>>>
>>> getString()
>>>
>>> Description: Finds a string between pre and post delimiters and stores into
>>> a given buffer. The string will be truncated to fit the buffer length. The
>>> string is terminated by the first character following the preString that
>>> matches the first character of the terminator.
>>>
>>> Syntax: getString(preString, postString, buffer, length)
>>>
>>> Parameters: preString: a string that precedes the desired target string,
>>> following characters this are appended to the given buffer until the first
>>> character of postString is encountered, length: the size of buffer.
>>>
>>> Returns: Returns the number of characters placed in the buffer (0 means no
>>> valid data found)
>>>
>>> Example:
>>>
>>> char buffer[8]; // must be big enough to hold target string and
>>> terminating null
>>>
>>> // get a string between "<" and ">"
>>>
>>> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>>>
>>> {
>>>
>>> Serial.print("found ");
>>>
>>> Serial.println(buffer);
>>>
>>> }
>>>
>>> This will print �target� if the stream contains ��
>>>
>>> Characters before the �<� are discarded. The next character read from the
>>> stream will be the character following �>�
>>>
>>>
>>>
>>> -----
>>>
>>> This functionality has worked quite well for me but feel free to suggest
>>> improvements to the API, implementation or documentation.
>>>
>>> You can run the examples in the zip by copying Stream.h and Stream.cpp to
>>> the cores > arduino directory of the 1.0 beta.
>>>
>>>
>>>
>>> Michael Margolis
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

>the process is something like this:

>find()
>appendToTempBufferUntil()
>copy temp buffer to working variable or whatever

Could you use the getString method to do what you want?

getString( "", "", tempBuffer, sizeof(tempBuffer));

Michael

-----Original Message-----
From: Tom Igoe
Sent: Tuesday, May 31, 2011 6:44 PM
To: Michael Margolis
Cc: David A. Mellis ;
Subject: Re: [Developers] Adding parsing to Stream

Yeah, I know what you mean, it's a challenge,because there's so much stuff
you have to filter through to get to the part you want, much of the time.
Ideally, I guess the process is something like this:

find()
appendToTempBufferUntil()
copy temp buffer to working variable or whatever

I guess you could do a saveUntil(), but that would require a dynamically
allocatable temp buffer, and that's ugly. Probably better to just leave it
as is and do the combo of find() then read().

t.



On May 31, 2011, at 12:56 PM, Michael Margolis wrote:

> Tom, thanks for your comments.
>
> The challenge with using peek is handling cases where the number of
> characters in the document is greater than the buffer size. For example,
> the google finance example I use in the Arduino Cookbook extracts a stock
> price that is typically 50k characters beyond the document head. If
> useful I could add a peek(target) and peekUntil(targetStr, terminateStr)
> that returned the location of the given target in the buffer or -1 if not
> found in the buffer. But I wonder if this functionality would be useful if
> the data of interest exceeded the stream buffer size.
>
> Michael
>
> -----Original Message----- From: Tom Igoe
> Sent: Tuesday, May 31, 2011 4:53 PM
> To: Michael Margolis
> Cc: David A. Mellis ;
> Subject: Re: [Developers] Adding parsing to Stream
>
> i have been using TextFinder in the rewrite of Making Things talk and it's
> very handy. I'd love to see it in the core. I'd love to see find() and
> findUntil() use peek() perhaps, so that you can do multiple finds() on a
> stream. I've had some cases where I needed to switch from find() to a
> character based search. For example, if I'm looking for
>
> This is what I want
>
> I have to find() for the then read() until I see a <. Maybe I'm
> missing something, but it'd be nice to just slurp everything between
> and
>
> t.
>
>
>
>
>
> On May 31, 2011, at 10:32 AM, Michael Margolis wrote:
>
>> David, thanks for your comments.
>>
>>> Would you want to be able to specify more than one skip character?
>>> Do we need some way to distinguish between a timeout and a real value of
>>> 0?
>> Good questions, I am interested to hear what the people here think will
>> be most useful.
>> These things are easy to add but I would like to avoid bloating the API
>> with features that are not likely to be used.
>>
>>> For getFloat(), do we need a way to specify the character that
>>> separates the whole and fractional part of the number (e.g. supporting
>>> European formatting like 1.234,56)?
>> This could be added by specifying the decimal point character to a user
>> supplied character such as a comma (default is '.')
>> If locale specific configuration was needed in elsewhere in Arduino
>> functionality I would suggest having a locale.h file that contains
>> defines for things like this. But a Stream method, perhaps named
>> setDecimalPointChar() could be added if this functionality was only
>> needed in this API.
>>
>>> I'd suggest using getInt() and getFloat() ...
>> getValue seemed less techy but I take the point about consistency.
>>
>> I would be interested to hear others views on these and anything else
>> regarding this API.
>>
>> Michael
>>
>>
>> -----Original Message----- From: David A. Mellis
>> Sent: Tuesday, May 31, 2011 2:30 PM
>> To: Michael Margolis
>> Cc:
>> Subject: Re: [Developers] Adding parsing to Stream
>>
>> Thanks for the proposal!
>>
>> Does anyone have any suggestions for the API or specific use cases we
>> should keep in mind?
>>
>> In particular, I'm curious about the precise semantics for the
>> getValue() functions. Would you want to be able to specify more than
>> one skip character? Do we need some way to distinguish between a
>> timeout and a real value of 0?
>>
>> For getFloat(), do we need a way to specify the character that
>> separates the whole and fractional part of the number (e.g. supporting
>> European formatting like 1.234,56)?
>>
>> I'd suggest using getInt() and getFloat() for consistency with the
>> names of the data types, even if we actually return longs and doubles.
>> The word "value" seems very generic.
>>
>> David
>>
>> On Tue, May 31, 2011 at 7:27 AM, Michael Margolis
>> <> wrote:
>>> This is a proposal for enhancements to Stream to support the finding and
>>> extracting numeric and string data. The parsing is done character by
>>> character so no intermediate static or dynamic buffer is required for
>>> the
>>> incoming data.
>>>
>>>
>>>
>>> The code is based on the TextFinder library
>>> (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with
>>> help
>>> from Philip Lindsay) for the Arduino Cookbook.
>>>
>>>
>>>
>>> The attached zip file contains 1.0 core Stream source code that
>>> implements
>>> the API described below. Examples showing use with Serial and Ethernet
>>> are
>>> also included. Please feel free to suggest improvements to the API,
>>> implementation or examples.
>>>
>>>
>>>
>>> Here are the currently supported methods:
>>>
>>>
>>>
>>> setTimeout()
>>>
>>> Description: Sets the maximum number of seconds to wait for a character
>>> from the stream. The default value is 5 seconds. The timeout value is
>>> used
>>> by all of the other parsing methods and could be useful for other
>>> stream
>>> functions.
>>>
>>> Syntax: setTimeout( timeout)
>>>
>>> Parameters: timout: the maximum number of seconds to wait.
>>>
>>> Returns: nothing
>>>
>>> Example:
>>>
>>> setTimeout(10); // set timeout to ten seconds
>>>
>>>
>>>
>>> find()
>>>
>>> Description: Reads data from the stream until the target string is
>>> found
>>>
>>> Syntax: find(targetStr)
>>>
>>> Parameters: targetStr is the null terminated C character string to
>>> search
>>> for
>>>
>>> Returns: true if target string is found. All characters up to and
>>> including
>>> the last character in targetStr will be flushed. Returns false if the
>>> targetStr is not found within the timeout period.
>>>
>>> Example:
>>>
>>> if( client.find("GET /") ) { // search for 'GET'
>>>
>>> // here if the string "GET /" is found in the stream
>>>
>>> else
>>>
>>> // here if find method timed out (stream buffer is empty)
>>>
>>>
>>>
>>> findUntil()
>>>
>>> Description: As find but search ends if the terminate string is found.
>>> Useful to stop searches at the end of a line or at a string token.
>>>
>>> Syntax: findUntil(targetStr, terminateStr)
>>>
>>> Parameters: targetStr is the null terminated C string to search for,
>>> terminateStr is the null terminated C string that will terminate the
>>> search
>>> if found in the stream.
>>>
>>> Returns: true if targetStr is found, returns false if timeout or
>>> terminateStr is encountered.
>>>
>>> Example:
>>>
>>> // search to the end of line for �pin�
>>>
>>> if(client.findUntil("pin", "\n\r"))
>>>
>>> // here if �pin� is found in the current line
>>>
>>> else
>>>
>>> // stream buffer is at the beginning of the next line
>>>
>>>
>>>
>>> Note that a binary variants of find and findUntil could be implemented
>>> if
>>> necessary for searching for binary data. Parameters would be a pointer
>>> to a
>>> byte array and the length of the target array. This would be
>>> functionally
>>> equivalent to the above but would handle null characters in the target
>>> and
>>> terminate strings.
>>>
>>>
>>>
>>>
>>>
>>> getValue()
>>>
>>> Description: Returns the first valid (long) integer value from the
>>> current
>>> position. Initial characters that are not ASCII digits (or a minus sign)
>>> are
>>> ignored. The value is terminated by the first character that is not a
>>> digit
>>> following one or more valid digits.
>>>
>>> Syntax: getvalue()
>>>
>>> Parameters: none
>>>
>>> Returns: long integer value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // this fragment will extract and print three comma separated integer
>>> values
>>>
>>> int value;
>>>
>>> for(int Index = 0; Index < 3; Index ++)
>>>
>>> {
>>>
>>> value = Serial.getValue(); // get a numeric value
>>>
>>> Serial.println(value);
>>>
>>> }
>>>
>>>
>>>
>>> getValue(skipChar)
>>>
>>> Description: As above but the given skipChar is ignored. This allows
>>> format
>>> characters (typically commas) in values to be ignored
>>>
>>> Syntax: getvalue(skipChar)
>>>
>>> Parameters: skipChar: a character to ignore
>>>
>>> Returns: long integer value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12,345 terminated with a newline
>>>
>>> int value;
>>>
>>> value = Serial.getValue(','); // ignore commas
>>>
>>> Serial.println(value); // prints 12345
>>>
>>>
>>>
>>>
>>>
>>> getFloat()
>>>
>>> Description: float version of getValue.
>>>
>>> Syntax: getFloat()
>>>
>>> Parameters: none
>>>
>>> Returns: : floating point value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12.34 terminated with a newline
>>>
>>> float value;
>>>
>>> value = Serial.getFloat();
>>>
>>> Serial.println(value); // prints 12.34
>>>
>>>
>>>
>>>
>>>
>>> getFloat(skipChar)
>>>
>>> Description: As above but the given skipChar is ignored. This allows
>>> format
>>> characters (typically commas) in values to be ignored
>>>
>>> Syntax: getFloat(skipChar)
>>>
>>> Parameters: skipChar: a character to ignore
>>>
>>> Returns: floating point value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12,345.67 terminated with a newline
>>>
>>> float value;
>>>
>>> value = Serial.getFloat();
>>>
>>> Serial.println(value); // prints 12345.67
>>>
>>>
>>>
>>> getString()
>>>
>>> Description: Finds a string between pre and post delimiters and stores
>>> into
>>> a given buffer. The string will be truncated to fit the buffer length.
>>> The
>>> string is terminated by the first character following the preString that
>>> matches the first character of the terminator.
>>>
>>> Syntax: getString(preString, postString, buffer, length)
>>>
>>> Parameters: preString: a string that precedes the desired target string,
>>> following characters this are appended to the given buffer until the
>>> first
>>> character of postString is encountered, length: the size of buffer.
>>>
>>> Returns: Returns the number of characters placed in the buffer (0 means
>>> no
>>> valid data found)
>>>
>>> Example:
>>>
>>> char buffer[8]; // must be big enough to hold target string and
>>> terminating null
>>>
>>> // get a string between "<" and ">"
>>>
>>> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>>>
>>> {
>>>
>>> Serial.print("found ");
>>>
>>> Serial.println(buffer);
>>>
>>> }
>>>
>>> This will print �target� if the stream contains ��
>>>
>>> Characters before the �<� are discarded. The next character read from
>>> the
>>> stream will be the character following �>�
>>>
>>>
>>>
>>> -----
>>>
>>> This functionality has worked quite well for me but feel free to suggest
>>> improvements to the API, implementation or documentation.
>>>
>>> You can run the examples in the zip by copying Stream.h and Stream.cpp
>>> to
>>> the cores > arduino directory of the 1.0 beta.
>>>
>>>
>>>
>>> Michael Margolis
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc
)

It's lovely to see TextFinder getting nearer to inclusion in the core
distribution - I've been using it with Internet stuff for a while now
and it's really handy.

The only suggestions I'd make are possibly for Stream to include a
function to indicate if the end of the stream has been reached
(something like "bool eof();"). The default implementation would return
false, but stream that have an end, such as TCP or a file, could return
true when the end is reached. That would let the TextFinder
functionality give up sooner rather than wait for a timeout (which on
Internet operations /should/ be a noticeably high amount of time)

And secondly, have case insensitive versions of the matching. Although
you quickly get into problems with character sets and languages and such
there...

Neither of those suggestions should stop us from adding it now, as
they're just additions to the API to extend the functionality.

Cheers,

Adrian.

On 31/05/11 12:27, Michael Margolis wrote:
>
> This is a proposal for enhancements to Stream to support the finding
> and extracting numeric and string data. The parsing is done character
> by character so no intermediate static or dynamic buffer is required
> for the incoming data.
>
> The code is based on the TextFinder library
> (http://www.arduino.cc/playground/Code/TextFinder) that Iwrote (with
> help from Philip Lindsay) for the Arduino Cookbook.
>
> The attached zip file contains 1.0 core Stream source code that
> implements the API described below. Examples showing use with Serial
> and Ethernet are also included. Please feel free to suggest
> improvements to the API, implementation or examples.
>
> Here are the currently supported methods:
>
> *setTimeout()*
>
> Description:Sets the maximum number of seconds to wait for a character
> from the stream. The default value is 5 seconds. The timeout value is
> used by all of the other parsing methods and could be useful for
> other stream functions.
>
> Syntax: setTimeout( timeout)
>
> Parameters: timout: the maximum number of seconds to wait.
>
> Returns: nothing
>
> Example:
>
> setTimeout(10); // set timeout to ten seconds
>
> *find()*
>
> Description:Reads data from the stream until the target string is found
>
> Syntax: find(targetStr)
>
> Parameters: targetStr is the null terminated C character string to
> search for
>
> Returns: true if target string is found. All characters up to and
> including the last character in targetStr will be flushed. Returns
> false if the targetStr is not found within the timeout period.
>
> Example:
>
> if( client.find("GET /") ) {// search for 'GET'
>
> // here if the string "GET /" is found in the stream
>
> else
>
> // here if find method timed out (stream buffer is empty)
>
> *findUntil()*
>
> Description: As find but search ends if the terminate string is
> found. Useful to stop searches at the end of a line or at a string token.
>
> Syntax: findUntil(targetStr, terminateStr)
>
> Parameters: targetStr is the null terminated C string to search for,
> terminateStr is the null terminated C string that will terminate the
> search if found in the stream.
>
> Returns: true if targetStr is found, returns false if timeout or
> terminateStr is encountered.
>
> Example:
>
> // search to the end of line for “pin”
>
> if(client.findUntil("pin", "\n\r"))
>
> // here if “pin” is found in the current line
>
> else
>
> // stream buffer is at the beginning of the next line
>
> Note that a binary variants of find and findUntil could be implemented
> if necessary for searching for binary data. Parameters would be a
> pointer to a byte array and the length of the target array. This would
> be functionally equivalent to the above but would handle null
> characters in the target and terminate strings.
>
> *getValue()*
>
> Description:Returns the first valid (long) integer value from the
> current position. Initial characters that are not ASCII digits (or a
> minus sign) are ignored. The value is terminated by the first
> character that is not a digit following one or more valid digits.
>
> Syntax: getvalue()
>
> Parameters: none
>
> Returns: long integer value or 0 if timeout.
>
> Example:
>
> // this fragment will extract and print three comma separated integer
> values
>
> int value;
>
> for(int Index = 0; Index< 3; Index ++)
>
> {
>
> value = Serial.getValue(); // get a numeric value
>
> Serial.println(value);
>
> }
>
> *getValue(skipChar)*
>
> Description: As above but the given skipChar is ignored. This allows
> format characters (typically commas) in values to be ignored
>
> Syntax: getvalue(skipChar)
>
> Parameters:skipChar: a character to ignore
>
> Returns: long integer value or 0 if timeout.
>
> Example:
>
> // send $12,345 terminated with a newline
>
> int value;
>
> value = Serial.getValue(','); // ignore commas
>
> Serial.println(value);// prints 12345
>
> *getFloat()*
>
> Description:float version of getValue.
>
> Syntax: getFloat()
>
> Parameters: none
>
> Returns: : floating point value or 0 if timeout.
>
> Example:
>
> // send $12.34 terminated with a newline
>
> float value;
>
> value = Serial.getFloat();
>
> Serial.println(value);// prints 12.34
>
> *getFloat(skipChar)*
>
> Description:As above but the given skipChar is ignored. This allows
> format characters (typically commas) in values to be ignored
>
> Syntax: getFloat(skipChar)
>
> Parameters: skipChar: a character to ignore
>
> Returns:floating point value or 0 if timeout.
>
> Example:
>
> // send $12,345.67 terminated with a newline
>
> float value;
>
> value = Serial.getFloat();
>
> Serial.println(value);// prints 12345.67
>
> *getString()*
>
> Description: Finds a string between pre and post delimiters and
> stores into a given buffer. The string will be truncated to fit the
> buffer length. The string is terminated by the first character
> following the preString that matches the first character of the
> terminator.
>
> Syntax: getString(preString, postString, buffer,length)
>
> Parameters: preString: a string that precedes the desired target
> string, following characters this are appended to the given buffer
> until the first character of postString is encountered, length: the
> size of buffer.
>
> Returns:Returns the number of characters placed in the buffer (0 means
> no valid data found)
>
> Example:
>
> char buffer[8]; // must be big enough to hold target string and
> terminating null
>
> // get a string between "<" and ">"
>
> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>
> {
>
> Serial.print("found ");
>
> Serial.println(buffer);
>
> }
>
> This will print ‘target’ if the stream contains “”
>
> Characters before the ‘<’ are discarded. The next character read from
> the stream will be the character following ‘>’
>
> -----
>
> This functionality has worked quite well for me but feel free to
> suggest improvements to the API, implementation or documentation.
>
> You can run the examples in the zip by copying Stream.h and Stream.cpp
> to the cores > arduino directory of the 1.0 beta.
>
> Michael Margolis
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

Hi Greg,

I'd be interested in suggestions or just more testing of the HTTP
library that I've been building up for a while now. I've just pushed
the version using the new Ethernet API to github at
. It parses the response code,
handles the Content-Length header if there is one, and can do basic
authentication too.

It doesn't use TextFinder (yet) as I didn't want to add another
dependency to it, but as it's derived from Stream then you can use
TextFinder to parse the body content once the headers are processed and
that's what I usually end up doing :-)

Cheers,

Adrian.

On 31/05/11 15:53, Greg Borenstein wrote:
> Thus work looks great! One use case I've been thinking a lot about is HTTP response parsing. I'm seeing more and more users start playing with the Ethernet library and nearly all immediately run into a wall when trying to parse the incoming requests and responses into useful header data (i.e. response code and, secondarily, other header values) and a body whose payload they can also parse to extract their actual data.
>
> I've been thinking about putting together a library that would wrap the Ethernet library a bit to provide a much simpler API that didn't ask the user to do any parsing or stream handling except possibly parsing their own custom format in the body.
>
> This parsing API seems like a great toolkit to build that library with.
>
> -- Greg
>
> On May 31, 2011, at 10:32 AM, "Michael Margolis"<> wrote:
>
>> David, thanks for your comments.
>>
>>> Would you want to be able to specify more than one skip character?
>>> Do we need some way to distinguish between a timeout and a real value of 0?
>> Good questions, I am interested to hear what the people here think will be most useful.
>> These things are easy to add but I would like to avoid bloating the API with features that are not likely to be used.
>>
>>> For getFloat(), do we need a way to specify the character that
>>> separates the whole and fractional part of the number (e.g. supporting
>>> European formatting like 1.234,56)?
>> This could be added by specifying the decimal point character to a user supplied character such as a comma (default is '.')
>> If locale specific configuration was needed in elsewhere in Arduino functionality I would suggest having a locale.h file that contains defines for things like this. But a Stream method, perhaps named setDecimalPointChar() could be added if this functionality was only needed in this API.
>>
>>> I'd suggest using getInt() and getFloat() ...
>> getValue seemed less techy but I take the point about consistency.
>>
>> I would be interested to hear others views on these and anything else regarding this API.
>>
>> Michael
>>
>>
>> -----Original Message----- From: David A. Mellis
>> Sent: Tuesday, May 31, 2011 2:30 PM
>> To: Michael Margolis
>> Cc:
>> Subject: Re: [Developers] Adding parsing to Stream
>>
>> Thanks for the proposal!
>>
>> Does anyone have any suggestions for the API or specific use cases we
>> should keep in mind?
>>
>> In particular, I'm curious about the precise semantics for the
>> getValue() functions. Would you want to be able to specify more than
>> one skip character? Do we need some way to distinguish between a
>> timeout and a real value of 0?
>>
>> For getFloat(), do we need a way to specify the character that
>> separates the whole and fractional part of the number (e.g. supporting
>> European formatting like 1.234,56)?
>>
>> I'd suggest using getInt() and getFloat() for consistency with the
>> names of the data types, even if we actually return longs and doubles.
>> The word "value" seems very generic.
>>
>> David
>>
>> On Tue, May 31, 2011 at 7:27 AM, Michael Margolis
>> <> wrote:
>>> This is a proposal for enhancements to Stream to support the finding and
>>> extracting numeric and string data. The parsing is done character by
>>> character so no intermediate static or dynamic buffer is required for the
>>> incoming data.
>>>
>>>
>>>
>>> The code is based on the TextFinder library
>>> (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with help
>>> from Philip Lindsay) for the Arduino Cookbook.
>>>
>>>
>>>
>>> The attached zip file contains 1.0 core Stream source code that implements
>>> the API described below. Examples showing use with Serial and Ethernet are
>>> also included. Please feel free to suggest improvements to the API,
>>> implementation or examples.
>>>
>>>
>>>
>>> Here are the currently supported methods:
>>>
>>>
>>>
>>> setTimeout()
>>>
>>> Description: Sets the maximum number of seconds to wait for a character
>>> from the stream. The default value is 5 seconds. The timeout value is used
>>> by all of the other parsing methods and could be useful for other stream
>>> functions.
>>>
>>> Syntax: setTimeout( timeout)
>>>
>>> Parameters: timout: the maximum number of seconds to wait.
>>>
>>> Returns: nothing
>>>
>>> Example:
>>>
>>> setTimeout(10); // set timeout to ten seconds
>>>
>>>
>>>
>>> find()
>>>
>>> Description: Reads data from the stream until the target string is found
>>>
>>> Syntax: find(targetStr)
>>>
>>> Parameters: targetStr is the null terminated C character string to search
>>> for
>>>
>>> Returns: true if target string is found. All characters up to and including
>>> the last character in targetStr will be flushed. Returns false if the
>>> targetStr is not found within the timeout period.
>>>
>>> Example:
>>>
>>> if( client.find("GET /") ) { // search for 'GET'
>>>
>>> // here if the string "GET /" is found in the stream
>>>
>>> else
>>>
>>> // here if find method timed out (stream buffer is empty)
>>>
>>>
>>>
>>> findUntil()
>>>
>>> Description: As find but search ends if the terminate string is found.
>>> Useful to stop searches at the end of a line or at a string token.
>>>
>>> Syntax: findUntil(targetStr, terminateStr)
>>>
>>> Parameters: targetStr is the null terminated C string to search for,
>>> terminateStr is the null terminated C string that will terminate the search
>>> if found in the stream.
>>>
>>> Returns: true if targetStr is found, returns false if timeout or
>>> terminateStr is encountered.
>>>
>>> Example:
>>>
>>> // search to the end of line for “pin”
>>>
>>> if(client.findUntil("pin", "\n\r"))
>>>
>>> // here if “pin” is found in the current line
>>>
>>> else
>>>
>>> // stream buffer is at the beginning of the next line
>>>
>>>
>>>
>>> Note that a binary variants of find and findUntil could be implemented if
>>> necessary for searching for binary data. Parameters would be a pointer to a
>>> byte array and the length of the target array. This would be functionally
>>> equivalent to the above but would handle null characters in the target and
>>> terminate strings.
>>>
>>>
>>>
>>>
>>>
>>> getValue()
>>>
>>> Description: Returns the first valid (long) integer value from the current
>>> position. Initial characters that are not ASCII digits (or a minus sign) are
>>> ignored. The value is terminated by the first character that is not a digit
>>> following one or more valid digits.
>>>
>>> Syntax: getvalue()
>>>
>>> Parameters: none
>>>
>>> Returns: long integer value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // this fragment will extract and print three comma separated integer
>>> values
>>>
>>> int value;
>>>
>>> for(int Index = 0; Index< 3; Index ++)
>>>
>>> {
>>>
>>> value = Serial.getValue(); // get a numeric value
>>>
>>> Serial.println(value);
>>>
>>> }
>>>
>>>
>>>
>>> getValue(skipChar)
>>>
>>> Description: As above but the given skipChar is ignored. This allows format
>>> characters (typically commas) in values to be ignored
>>>
>>> Syntax: getvalue(skipChar)
>>>
>>> Parameters: skipChar: a character to ignore
>>>
>>> Returns: long integer value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12,345 terminated with a newline
>>>
>>> int value;
>>>
>>> value = Serial.getValue(','); // ignore commas
>>>
>>> Serial.println(value); // prints 12345
>>>
>>>
>>>
>>>
>>>
>>> getFloat()
>>>
>>> Description: float version of getValue.
>>>
>>> Syntax: getFloat()
>>>
>>> Parameters: none
>>>
>>> Returns: : floating point value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12.34 terminated with a newline
>>>
>>> float value;
>>>
>>> value = Serial.getFloat();
>>>
>>> Serial.println(value); // prints 12.34
>>>
>>>
>>>
>>>
>>>
>>> getFloat(skipChar)
>>>
>>> Description: As above but the given skipChar is ignored. This allows format
>>> characters (typically commas) in values to be ignored
>>>
>>> Syntax: getFloat(skipChar)
>>>
>>> Parameters: skipChar: a character to ignore
>>>
>>> Returns: floating point value or 0 if timeout.
>>>
>>> Example:
>>>
>>> // send $12,345.67 terminated with a newline
>>>
>>> float value;
>>>
>>> value = Serial.getFloat();
>>>
>>> Serial.println(value); // prints 12345.67
>>>
>>>
>>>
>>> getString()
>>>
>>> Description: Finds a string between pre and post delimiters and stores into
>>> a given buffer. The string will be truncated to fit the buffer length. The
>>> string is terminated by the first character following the preString that
>>> matches the first character of the terminator.
>>>
>>> Syntax: getString(preString, postString, buffer, length)
>>>
>>> Parameters: preString: a string that precedes the desired target string,
>>> following characters this are appended to the given buffer until the first
>>> character of postString is encountered, length: the size of buffer.
>>>
>>> Returns: Returns the number of characters placed in the buffer (0 means no
>>> valid data found)
>>>
>>> Example:
>>>
>>> char buffer[8]; // must be big enough to hold target string and
>>> terminating null
>>>
>>> // get a string between "<" and">"
>>>
>>> if(client.getString( "<",">", buffer, sizeof(buffer) ))
>>>
>>> {
>>>
>>> Serial.print("found ");
>>>
>>> Serial.println(buffer);
>>>
>>> }
>>>
>>> This will print ‘target’ if the stream contains “”
>>>
>>> Characters before the ‘<’ are discarded. The next character read from the
>>> stream will be the character following ‘>’
>>>
>>>
>>>
>>> -----
>>>
>>> This functionality has worked quite well for me but feel free to suggest
>>> improvements to the API, implementation or documentation.
>>>
>>> You can run the examples in the zip by copying Stream.h and Stream.cpp to
>>> the cores> arduino directory of the 1.0 beta.
>>>
>>>
>>>
>>> Michael Margolis
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

I'd call it end() instead of eof(). OR something else more descriptive to someone who's never heard the term eof. Unless you meant "oeuf", or "egg", which might confuse French users.

t.

On May 31, 2011, at 7:10 PM, Adrian McEwen wrote:

> It's lovely to see TextFinder getting nearer to inclusion in the core distribution - I've been using it with Internet stuff for a while now and it's really handy.
>
> The only suggestions I'd make are possibly for Stream to include a function to indicate if the end of the stream has been reached (something like "bool eof();"). The default implementation would return false, but stream that have an end, such as TCP or a file, could return true when the end is reached. That would let the TextFinder functionality give up sooner rather than wait for a timeout (which on Internet operations /should/ be a noticeably high amount of time)
>
> And secondly, have case insensitive versions of the matching. Although you quickly get into problems with character sets and languages and such there...
>
> Neither of those suggestions should stop us from adding it now, as they're just additions to the API to extend the functionality.
>
> Cheers,
>
> Adrian.
>
> On 31/05/11 12:27, Michael Margolis wrote:
>>
>> This is a proposal for enhancements to Stream to support the finding and extracting numeric and string data. The parsing is done character by character so no intermediate static or dynamic buffer is required for the incoming data.
>>
>> The code is based on the TextFinder library (http://www.arduino.cc/playground/Code/TextFinder) that I wrote (with help from Philip Lindsay) for the Arduino Cookbook.
>>
>> The attached zip file contains 1.0 core Stream source code that implements the API described below. Examples showing use with Serial and Ethernet are also included. Please feel free to suggest improvements to the API, implementation or examples.
>>
>> Here are the currently supported methods:
>>
>> setTimeout()
>> Description: Sets the maximum number of seconds to wait for a character from the stream. The default value is 5 seconds. The timeout value is used by all of the other parsing methods and could be useful for other stream functions.
>> Syntax: setTimeout( timeout)
>> Parameters: timout: the maximum number of seconds to wait.
>> Returns: nothing
>> Example:
>> setTimeout(10); // set timeout to ten seconds
>>
>> find()
>> Description: Reads data from the stream until the target string is found
>> Syntax: find(targetStr)
>> Parameters: targetStr is the null terminated C character string to search for
>> Returns: true if target string is found. All characters up to and including the last character in targetStr will be flushed. Returns false if the targetStr is not found within the timeout period.
>> Example:
>> if( client.find("GET /") ) { // search for 'GET'
>> // here if the string "GET /" is found in the stream
>> else
>> // here if find method timed out (stream buffer is empty)
>>
>> findUntil()
>> Description: As find but search ends if the terminate string is found. Useful to stop searches at the end of a line or at a string token.
>> Syntax: findUntil(targetStr, terminateStr)
>> Parameters: targetStr is the null terminated C string to search for, terminateStr is the null terminated C string that will terminate the search if found in the stream.
>> Returns: true if targetStr is found, returns false if timeout or terminateStr is encountered.
>> Example:
>> // search to the end of line for �pin�
>> if(client.findUntil("pin", "\n\r"))
>> // here if �pin� is found in the current line
>> else
>> // stream buffer is at the beginning of the next line
>>
>> Note that a binary variants of find and findUntil could be implemented if necessary for searching for binary data. Parameters would be a pointer to a byte array and the length of the target array. This would be functionally equivalent to the above but would handle null characters in the target and terminate strings.
>>
>>
>> getValue()
>> Description: Returns the first valid (long) integer value from the current position. Initial characters that are not ASCII digits (or a minus sign) are ignored. The value is terminated by the first character that is not a digit following one or more valid digits.
>> Syntax: getvalue()
>> Parameters: none
>> Returns: long integer value or 0 if timeout.
>> Example:
>> // this fragment will extract and print three comma separated integer values
>> int value;
>> for(int Index = 0; Index < 3; Index ++)
>> {
>> value = Serial.getValue(); // get a numeric value
>> Serial.println(value);
>> }
>>
>> getValue(skipChar)
>> Description: As above but the given skipChar is ignored. This allows format characters (typically commas) in values to be ignored
>> Syntax: getvalue(skipChar)
>> Parameters: skipChar: a character to ignore
>> Returns: long integer value or 0 if timeout.
>> Example:
>> // send $12,345 terminated with a newline
>> int value;
>> value = Serial.getValue(','); // ignore commas
>> Serial.println(value); // prints 12345
>>
>>
>> getFloat()
>> Description: float version of getValue.
>> Syntax: getFloat()
>> Parameters: none
>> Returns: : floating point value or 0 if timeout.
>> Example:
>> // send $12.34 terminated with a newline
>> float value;
>> value = Serial.getFloat();
>> Serial.println(value); // prints 12.34
>>
>>
>> getFloat(skipChar)
>> Description: As above but the given skipChar is ignored. This allows format characters (typically commas) in values to be ignored
>> Syntax: getFloat(skipChar)
>> Parameters: skipChar: a character to ignore
>> Returns: floating point value or 0 if timeout.
>> Example:
>> // send $12,345.67 terminated with a newline
>> float value;
>> value = Serial.getFloat();
>> Serial.println(value); // prints 12345.67
>>
>> getString()
>> Description: Finds a string between pre and post delimiters and stores into a given buffer. The string will be truncated to fit the buffer length. The string is terminated by the first character following the preString that matches the first character of the terminator.
>> Syntax: getString(preString, postString, buffer, length)
>> Parameters: preString: a string that precedes the desired target string, following characters this are appended to the given buffer until the first character of postString is encountered, length: the size of buffer.
>> Returns: Returns the number of characters placed in the buffer (0 means no valid data found)
>> Example:
>> char buffer[8]; // must be big enough to hold target string and terminating null
>> // get a string between "<" and ">"
>> if(client.getString( "<", ">", buffer, sizeof(buffer) ))
>> {
>> Serial.print("found ");
>> Serial.println(buffer);
>> }
>> This will print �target� if the stream contains ��
>> Characters before the �<� are discarded. The next character read from the stream will be the character following �>�
>>
>> -----
>> This functionality has worked quite well for me but feel free to suggest improvements to the API, implementation or documentation.
>> You can run the examples in the zip by copying Stream.h and Stream.cpp to the cores > arduino directory of the 1.0 beta.
>>
>> Michael Margolis
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc