We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe.

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
i have some code on github but its very much in progress


On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:

> Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:
>
> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>>> From: "David A. Mellis" <>
>>> Date: February 11, 2011 12:11:53 PM EST
>>> To: Tom Igoe <>
>>> Cc: Arduino Developers <>
>>> Subject: Re: [Developers] SD or SPI problem?
>>>
>>> One thing it could be is the fact that we open files in sync mode,
>>> which means all changes are immediately physically written to the SD
>>> card. This slows things done, but is more robust, because there
>>> aren't any buffered changes to lose if power dies, for example. You
>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>> then retry the sketch to see if it changes the speed. If that's the
>>> issue, we should be able to find a way to resolve it.
>>>
>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>
>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>
>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>
>>>> t.
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
i have some code on github but its very much in progress


On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:

> Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:
>
> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>>> From: "David A. Mellis" <>
>>> Date: February 11, 2011 12:11:53 PM EST
>>> To: Tom Igoe <>
>>> Cc: Arduino Developers <>
>>> Subject: Re: [Developers] SD or SPI problem?
>>>
>>> One thing it could be is the fact that we open files in sync mode,
>>> which means all changes are immediately physically written to the SD
>>> card. This slows things done, but is more robust, because there
>>> aren't any buffered changes to lose if power dies, for example. You
>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>> then retry the sketch to see if it changes the speed. If that's the
>>> issue, we should be able to find a way to resolve it.
>>>
>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>
>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>
>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>
>>>> t.
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

2011/2/22 Tom Igoe <>:
> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

What do you mean by turning it off and on? Is acceptable to have a
way to open the file in buffered or unbuffered mode, with an explicit
sync option for buffered mode? Or do you think we need to be able to
dynamically switch between buffered and unbuffered modes? If the
latter, why? Or maybe we can get away with simply using buffered mode
all the time, and having people sync when needed?

> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

I did this at one point during development, but it seemed unnecessary
given that you can delete the file before opening it. Would you want
to go back to having FILE_APPEND or FILE_REPLACE instead of
FILE_WRITE? Or distinguish the two some other way?

> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
>> >> Developers mailing list
>> >>
>> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>> >>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
i have some code on github but its very much in progress


On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:

> Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:
>
> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>>> From: "David A. Mellis" <>
>>> Date: February 11, 2011 12:11:53 PM EST
>>> To: Tom Igoe <>
>>> Cc: Arduino Developers <>
>>> Subject: Re: [Developers] SD or SPI problem?
>>>
>>> One thing it could be is the fact that we open files in sync mode,
>>> which means all changes are immediately physically written to the SD
>>> card. This slows things done, but is more robust, because there
>>> aren't any buffered changes to lose if power dies, for example. You
>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>> then retry the sketch to see if it changes the speed. If that's the
>>> issue, we should be able to find a way to resolve it.
>>>
>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>
>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>
>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>
>>>> t.
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

2011/2/22 Tom Igoe <>:
> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

What do you mean by turning it off and on? Is acceptable to have a
way to open the file in buffered or unbuffered mode, with an explicit
sync option for buffered mode? Or do you think we need to be able to
dynamically switch between buffered and unbuffered modes? If the
latter, why? Or maybe we can get away with simply using buffered mode
all the time, and having people sync when needed?

> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

I did this at one point during development, but it seemed unnecessary
given that you can delete the file before opening it. Would you want
to go back to having FILE_APPEND or FILE_REPLACE instead of
FILE_WRITE? Or distinguish the two some other way?

> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
>> >> Developers mailing list
>> >>
>> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>> >>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

That could be very useful. What do you have in mind for the API?

2011/2/22 Limor <>:
> somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
>  i have some code on github but its very much in progress
>
>
> On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:
>
>>    Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up.  But working on both, we ran across two important needs:
>>
>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card.  This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
i have some code on github but its very much in progress


On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:

> Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:
>
> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>>> From: "David A. Mellis" <>
>>> Date: February 11, 2011 12:11:53 PM EST
>>> To: Tom Igoe <>
>>> Cc: Arduino Developers <>
>>> Subject: Re: [Developers] SD or SPI problem?
>>>
>>> One thing it could be is the fact that we open files in sync mode,
>>> which means all changes are immediately physically written to the SD
>>> card. This slows things done, but is more robust, because there
>>> aren't any buffered changes to lose if power dies, for example. You
>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>> then retry the sketch to see if it changes the speed. If that's the
>>> issue, we should be able to find a way to resolve it.
>>>
>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>
>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>
>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>
>>>> t.
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

2011/2/22 Tom Igoe <>:
> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

What do you mean by turning it off and on? Is acceptable to have a
way to open the file in buffered or unbuffered mode, with an explicit
sync option for buffered mode? Or do you think we need to be able to
dynamically switch between buffered and unbuffered modes? If the
latter, why? Or maybe we can get away with simply using buffered mode
all the time, and having people sync when needed?

> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

I did this at one point during development, but it seemed unnecessary
given that you can delete the file before opening it. Would you want
to go back to having FILE_APPEND or FILE_REPLACE instead of
FILE_WRITE? Or distinguish the two some other way?

> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
>> >> Developers mailing list
>> >>
>> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>> >>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

That could be very useful. What do you have in mind for the API?

2011/2/22 Limor <>:
> somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
>  i have some code on github but its very much in progress
>
>
> On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:
>
>>    Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up.  But working on both, we ran across two important needs:
>>
>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card.  This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

Which is buffered mode? With O_SYNC on or off? It's much faster with it off, but it does mean you might lose some data if you lose power, right?

I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.

Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?

t.

On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:

> 2011/2/22 Tom Igoe <>:
>> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> What do you mean by turning it off and on? Is acceptable to have a
> way to open the file in buffered or unbuffered mode, with an explicit
> sync option for buffered mode? Or do you think we need to be able to
> dynamically switch between buffered and unbuffered modes? If the
> latter, why? Or maybe we can get away with simply using buffered mode
> all the time, and having people sync when needed?
>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> I did this at one point during development, but it seemed unnecessary
> given that you can delete the file before opening it. Would you want
> to go back to having FILE_APPEND or FILE_REPLACE instead of
> FILE_WRITE? Or distinguish the two some other way?
>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
i have some code on github but its very much in progress


On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:

> Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:
>
> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>>> From: "David A. Mellis" <>
>>> Date: February 11, 2011 12:11:53 PM EST
>>> To: Tom Igoe <>
>>> Cc: Arduino Developers <>
>>> Subject: Re: [Developers] SD or SPI problem?
>>>
>>> One thing it could be is the fact that we open files in sync mode,
>>> which means all changes are immediately physically written to the SD
>>> card. This slows things done, but is more robust, because there
>>> aren't any buffered changes to lose if power dies, for example. You
>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>> then retry the sketch to see if it changes the speed. If that's the
>>> issue, we should be able to find a way to resolve it.
>>>
>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>
>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>
>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>
>>>> t.
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

2011/2/22 Tom Igoe <>:
> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

What do you mean by turning it off and on? Is acceptable to have a
way to open the file in buffered or unbuffered mode, with an explicit
sync option for buffered mode? Or do you think we need to be able to
dynamically switch between buffered and unbuffered modes? If the
latter, why? Or maybe we can get away with simply using buffered mode
all the time, and having people sync when needed?

> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

I did this at one point during development, but it seemed unnecessary
given that you can delete the file before opening it. Would you want
to go back to having FILE_APPEND or FILE_REPLACE instead of
FILE_WRITE? Or distinguish the two some other way?

> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
>> >> Developers mailing list
>> >>
>> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>> >>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

That could be very useful. What do you have in mind for the API?

2011/2/22 Limor <>:
> somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
>  i have some code on github but its very much in progress
>
>
> On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:
>
>>    Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up.  But working on both, we ran across two important needs:
>>
>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card.  This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

Which is buffered mode? With O_SYNC on or off? It's much faster with it off, but it does mean you might lose some data if you lose power, right?

I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.

Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?

t.

On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:

> 2011/2/22 Tom Igoe <>:
>> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> What do you mean by turning it off and on? Is acceptable to have a
> way to open the file in buffered or unbuffered mode, with an explicit
> sync option for buffered mode? Or do you think we need to be able to
> dynamically switch between buffered and unbuffered modes? If the
> latter, why? Or maybe we can get away with simply using buffered mode
> all the time, and having people sync when needed?
>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> I did this at one point during development, but it seemed unnecessary
> given that you can delete the file before opening it. Would you want
> to go back to having FILE_APPEND or FILE_REPLACE instead of
> FILE_WRITE? Or distinguish the two some other way?
>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

By buffered, I mean with O_SYNC off, when writes are buffered in the
memory of the microcontroller before being written to the SD card.
The flush() command will force a sync.

2011/2/23 Tom Igoe <>:
> Which is buffered mode?  With O_SYNC on or off?  It's much faster with it off, but it does mean you might lose some data if you lose power, right?
>
> I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.
>
> Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?
>
> t.
>
> On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:
>
>> 2011/2/22 Tom Igoe <>:
>>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> What do you mean by turning it off and on?  Is acceptable to have a
>> way to open the file in buffered or unbuffered mode, with an explicit
>> sync option for buffered mode?  Or do you think we need to be able to
>> dynamically switch between buffered and unbuffered modes?  If the
>> latter, why?  Or maybe we can get away with simply using buffered mode
>> all the time, and having people sync when needed?
>>
>>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> I did this at one point during development, but it seemed unnecessary
>> given that you can delete the file before opening it.  Would you want
>> to go back to having FILE_APPEND or FILE_REPLACE instead of
>> FILE_WRITE?  Or distinguish the two some other way?
>>
>>> t.
>>>
>>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>>
>>>> done!
>>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>>
>>>> m
>>>>
>>>>
>>>>
>>>> 2011/2/11 Tom Igoe <>
>>>> A possible solution to the slow response from the SD library:
>>>>
>>>> t.
>>>>
>>>>
>>>> Begin forwarded message:
>>>>
>>>>> From: "David A. Mellis" <>
>>>>> Date: February 11, 2011 12:11:53 PM EST
>>>>> To: Tom Igoe <>
>>>>> Cc: Arduino Developers <>
>>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>>
>>>>> One thing it could be is the fact that we open files in sync mode,
>>>>> which means all changes are immediately physically written to the SD
>>>>> card.  This slows things done, but is more robust, because there
>>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>>> issue, we should be able to find a way to resolve it.
>>>>>
>>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>>
>>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>>
>>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>>
>>>>>> t.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Developers mailing list
>>>>>>
>>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
i have some code on github but its very much in progress


On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:

> Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:
>
> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>>> From: "David A. Mellis" <>
>>> Date: February 11, 2011 12:11:53 PM EST
>>> To: Tom Igoe <>
>>> Cc: Arduino Developers <>
>>> Subject: Re: [Developers] SD or SPI problem?
>>>
>>> One thing it could be is the fact that we open files in sync mode,
>>> which means all changes are immediately physically written to the SD
>>> card. This slows things done, but is more robust, because there
>>> aren't any buffered changes to lose if power dies, for example. You
>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>> then retry the sketch to see if it changes the speed. If that's the
>>> issue, we should be able to find a way to resolve it.
>>>
>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>
>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>
>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>
>>>> t.
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

2011/2/22 Tom Igoe <>:
> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

What do you mean by turning it off and on? Is acceptable to have a
way to open the file in buffered or unbuffered mode, with an explicit
sync option for buffered mode? Or do you think we need to be able to
dynamically switch between buffered and unbuffered modes? If the
latter, why? Or maybe we can get away with simply using buffered mode
all the time, and having people sync when needed?

> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

I did this at one point during development, but it seemed unnecessary
given that you can delete the file before opening it. Would you want
to go back to having FILE_APPEND or FILE_REPLACE instead of
FILE_WRITE? Or distinguish the two some other way?

> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
>> >> Developers mailing list
>> >>
>> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>> >>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

That could be very useful. What do you have in mind for the API?

2011/2/22 Limor <>:
> somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
>  i have some code on github but its very much in progress
>
>
> On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:
>
>>    Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up.  But working on both, we ran across two important needs:
>>
>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card.  This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

Which is buffered mode? With O_SYNC on or off? It's much faster with it off, but it does mean you might lose some data if you lose power, right?

I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.

Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?

t.

On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:

> 2011/2/22 Tom Igoe <>:
>> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> What do you mean by turning it off and on? Is acceptable to have a
> way to open the file in buffered or unbuffered mode, with an explicit
> sync option for buffered mode? Or do you think we need to be able to
> dynamically switch between buffered and unbuffered modes? If the
> latter, why? Or maybe we can get away with simply using buffered mode
> all the time, and having people sync when needed?
>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> I did this at one point during development, but it seemed unnecessary
> given that you can delete the file before opening it. Would you want
> to go back to having FILE_APPEND or FILE_REPLACE instead of
> FILE_WRITE? Or distinguish the two some other way?
>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

By buffered, I mean with O_SYNC off, when writes are buffered in the
memory of the microcontroller before being written to the SD card.
The flush() command will force a sync.

2011/2/23 Tom Igoe <>:
> Which is buffered mode?  With O_SYNC on or off?  It's much faster with it off, but it does mean you might lose some data if you lose power, right?
>
> I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.
>
> Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?
>
> t.
>
> On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:
>
>> 2011/2/22 Tom Igoe <>:
>>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> What do you mean by turning it off and on?  Is acceptable to have a
>> way to open the file in buffered or unbuffered mode, with an explicit
>> sync option for buffered mode?  Or do you think we need to be able to
>> dynamically switch between buffered and unbuffered modes?  If the
>> latter, why?  Or maybe we can get away with simply using buffered mode
>> all the time, and having people sync when needed?
>>
>>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> I did this at one point during development, but it seemed unnecessary
>> given that you can delete the file before opening it.  Would you want
>> to go back to having FILE_APPEND or FILE_REPLACE instead of
>> FILE_WRITE?  Or distinguish the two some other way?
>>
>>> t.
>>>
>>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>>
>>>> done!
>>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>>
>>>> m
>>>>
>>>>
>>>>
>>>> 2011/2/11 Tom Igoe <>
>>>> A possible solution to the slow response from the SD library:
>>>>
>>>> t.
>>>>
>>>>
>>>> Begin forwarded message:
>>>>
>>>>> From: "David A. Mellis" <>
>>>>> Date: February 11, 2011 12:11:53 PM EST
>>>>> To: Tom Igoe <>
>>>>> Cc: Arduino Developers <>
>>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>>
>>>>> One thing it could be is the fact that we open files in sync mode,
>>>>> which means all changes are immediately physically written to the SD
>>>>> card.  This slows things done, but is more robust, because there
>>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>>> issue, we should be able to find a way to resolve it.
>>>>>
>>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>>
>>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>>
>>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>>
>>>>>> t.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Developers mailing list
>>>>>>
>>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

Oh, perfect, then I think just adding a #define for FILE_WRITE_BUFFERED vs FILE_WRITE will do the trick, and we can write up the differences. OK if i just make that change to the repo?

t.

On Feb 23, 2011, at 8:43 AM, David A. Mellis wrote:

> By buffered, I mean with O_SYNC off, when writes are buffered in the
> memory of the microcontroller before being written to the SD card.
> The flush() command will force a sync.
>
> 2011/2/23 Tom Igoe <>:
>> Which is buffered mode? With O_SYNC on or off? It's much faster with it off, but it does mean you might lose some data if you lose power, right?
>>
>> I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.
>>
>> Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?
>>
>> t.
>>
>> On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:
>>
>>> 2011/2/22 Tom Igoe <>:
>>>> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>>
>>> What do you mean by turning it off and on? Is acceptable to have a
>>> way to open the file in buffered or unbuffered mode, with an explicit
>>> sync option for buffered mode? Or do you think we need to be able to
>>> dynamically switch between buffered and unbuffered modes? If the
>>> latter, why? Or maybe we can get away with simply using buffered mode
>>> all the time, and having people sync when needed?
>>>
>>>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>>
>>> I did this at one point during development, but it seemed unnecessary
>>> given that you can delete the file before opening it. Would you want
>>> to go back to having FILE_APPEND or FILE_REPLACE instead of
>>> FILE_WRITE? Or distinguish the two some other way?
>>>
>>>> t.
>>>>
>>>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>>>
>>>>> done!
>>>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>>>
>>>>> m
>>>>>
>>>>>
>>>>>
>>>>> 2011/2/11 Tom Igoe <>
>>>>> A possible solution to the slow response from the SD library:
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> Begin forwarded message:
>>>>>
>>>>>> From: "David A. Mellis" <>
>>>>>> Date: February 11, 2011 12:11:53 PM EST
>>>>>> To: Tom Igoe <>
>>>>>> Cc: Arduino Developers <>
>>>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>>>
>>>>>> One thing it could be is the fact that we open files in sync mode,
>>>>>> which means all changes are immediately physically written to the SD
>>>>>> card. This slows things done, but is more robust, because there
>>>>>> aren't any buffered changes to lose if power dies, for example. You
>>>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>>>> then retry the sketch to see if it changes the speed. If that's the
>>>>>> issue, we should be able to find a way to resolve it.
>>>>>>
>>>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>>>
>>>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>>>
>>>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>>>
>>>>>>> t.
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Developers mailing list
>>>>>>>
>>>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.

With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
With the SD library: took 260 seconds to write the same image to the same SD card.

Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?

t.


_______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. One thing it could be is the fact that we open files in sync mode,
which means all changes are immediately physically written to the SD
card. This slows things done, but is more robust, because there
aren't any buffered changes to lose if power dies, for example. You
could try removing O_SYNC from the FILE_WRITE definition in SD.h and
then retry the sketch to see if it changes the speed. If that's the
issue, we should be able to find a way to resolve it.

On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>
> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> With the SD library:  took 260 seconds to write the same image to the same SD card.
>
> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>
> t.
>
>
> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:

#define FILE_READ O_READ
#define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
#define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)

Or we could add a function to turn the sync buffer on and off, like so:

SD.bufferWrite(ON);
or
SD.bufferWrite(OFF);

I'f imagine you'd call this right before calling SD.open()

Which seems better?


On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. another option is to pass it as an extra argument in the write() command
so you could speed up some writes (like fast data streams) but not all

On 2/18/2011 7:14 AM, Tom Igoe wrote:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe<>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis"<>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe<>
>> > Cc: Arduino Developers<>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card. This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example. You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed. If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library: took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
keep the safer but slower mode as the one people get if they don't
explicitly request the other?

I'm assuming that we don't need a way to dynamically switch between
buffered and unbuffered modes. Right? Calling the flush() function
will explicitly save all changes to the SD card if you're in buffered
mode.

Added to the Google Code issue list:
http://code.google.com/p/arduino/issues/detail?id=483

David

2011/2/18 Tom Igoe <>:
> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off.  See discussion below for the magnitude of the change.  Perhaps we could add an API option for turning it on or off?  Easiest might be to add a definition like so:
>
> #define FILE_READ O_READ
> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>
> Or we could add a function to turn the sync buffer on and off, like so:
>
> SD.bufferWrite(ON);
> or
> SD.bufferWrite(OFF);
>
> I'f imagine you'd call this right before calling SD.open()
>
> Which seems better?
>
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Interesting, but I think I'd rather just do it in the file open mode.

2011/2/18 Limor <>:
> another option is to pass it as an extra argument in the write() command so
> you could speed up some writes (like fast data streams) but not all
>
> On 2/18/2011 7:14 AM, Tom Igoe wrote:
>>
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD
>> library and found that you get a significant speed increase with O_SYNC off.
>>  See discussion below for the magnitude of the change.  Perhaps we could add
>> an API option for turning it on or off?  Easiest might be to add a
>> definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>>  done!
>>>  It is even faster dan Sparkfun! Yay arduino! you can see the screen shot
>>> fromt he serial and the picture as attached.
>>>
>>>  m
>>>
>>>
>>>
>>>  2011/2/11 Tom Igoe<>
>>>  A possible solution to the slow response from the SD library:
>>>
>>>  t.
>>>
>>>
>>>  Begin forwarded message:
>>>
>>>  >  From: "David A. Mellis"<>
>>>  >  Date: February 11, 2011 12:11:53 PM EST
>>>  >  To: Tom Igoe<>
>>>  >  Cc: Arduino Developers<>
>>>  >  Subject: Re: [Developers] SD or SPI problem?
>>>  >
>>>  >  One thing it could be is the fact that we open files in sync mode,
>>>  >  which means all changes are immediately physically written to the SD
>>>  >  card.  This slows things done, but is more robust, because there
>>>  >  aren't any buffered changes to lose if power dies, for example.  You
>>>  >  could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>  >  then retry the sketch to see if it changes the speed.  If that's the
>>>  >  issue, we should be able to find a way to resolve it.
>>>  >
>>>  >  On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe<>  wrote:
>>>  >>  We've been playing around with a LinkSprite camera here at ITP, and
>>> as part of it one of our researchers ran SparkFun's example that writes to
>>> an SD card via the MemoryCard library.  We took the same code, and changed
>>> it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>  >>
>>>  >>  With the MemoryCard library: Arduino Uno took approx. 6 seconds to
>>> write to an SD card, a 12K file.
>>>  >>  With the SD library:  took 260 seconds to write the same image to
>>> the same SD card.
>>>  >>
>>>  >>  Not sure if the problem is with the SD library, or the SPI library,
>>> but it bears some investigation.  I recall we discsussed SPI modes, but
>>> can't recall what we defaulted to. Anyone got an idea?
>>>  >>
>>>  >>  t.
>>>  >>
>>>  >>
>>>  >>  _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. I like Limor's idea too, but this seems like the easiest solution. I didn't realize that about the flush() command, that's good to know.

t.

On Feb 18, 2011, at 10:26 AM, David A. Mellis wrote:

> Maybe FILE_WRITE and FILE_WRITE_BUFFERED or something? That way, we
> keep the safer but slower mode as the one people get if they don't
> explicitly request the other?
>
> I'm assuming that we don't need a way to dynamically switch between
> buffered and unbuffered modes. Right? Calling the flush() function
> will explicitly save all changes to the SD card if you're in buffered
> mode.
>
> Added to the Google Code issue list:
> http://code.google.com/p/arduino/issues/detail?id=483
>
> David
>
> 2011/2/18 Tom Igoe <>:
>> So Mustafa Bagdatli did a test of the LinkSprite camera with the SD library and found that you get a significant speed increase with O_SYNC off. See discussion below for the magnitude of the change. Perhaps we could add an API option for turning it on or off? Easiest might be to add a definition like so:
>>
>> #define FILE_READ O_READ
>> #define FILE_WRITE_FAST (O_READ | O_WRITE | O_CREAT)
>> #define FILE_WRITE_SECURE (O_READ | O_WRITE | O_CREAT | O_SYNC)
>>
>> Or we could add a function to turn the sync buffer on and off, like so:
>>
>> SD.bufferWrite(ON);
>> or
>> SD.bufferWrite(OFF);
>>
>> I'f imagine you'd call this right before calling SD.open()
>>
>> Which seems better?
>>
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
___________________________________________________

Posted on the Developers mailing list. Go to http://arduino.cc/mailman/listinfo/developers_arduino.cc to subscribe. Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:

1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

t.

On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:

> done!
> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>
> m
>
>
>
> 2011/2/11 Tom Igoe <>
> A possible solution to the slow response from the SD library:
>
> t.
>
>
> Begin forwarded message:
>
> > From: "David A. Mellis" <>
> > Date: February 11, 2011 12:11:53 PM EST
> > To: Tom Igoe <>
> > Cc: Arduino Developers <>
> > Subject: Re: [Developers] SD or SPI problem?
> >
> > One thing it could be is the fact that we open files in sync mode,
> > which means all changes are immediately physically written to the SD
> > card. This slows things done, but is more robust, because there
> > aren't any buffered changes to lose if power dies, for example. You
> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
> > then retry the sketch to see if it changes the speed. If that's the
> > issue, we should be able to find a way to resolve it.
> >
> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
> >>
> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
> >> With the SD library: took 260 seconds to write the same image to the same SD card.
> >>
> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
> >>
> >> t.
> >>
> >>
> >> _______________________________________________
> >> Developers mailing list
> >>
> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
> >>
>
>
>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
i have some code on github but its very much in progress


On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:

> Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up. But working on both, we ran across two important needs:
>
> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>>> From: "David A. Mellis" <>
>>> Date: February 11, 2011 12:11:53 PM EST
>>> To: Tom Igoe <>
>>> Cc: Arduino Developers <>
>>> Subject: Re: [Developers] SD or SPI problem?
>>>
>>> One thing it could be is the fact that we open files in sync mode,
>>> which means all changes are immediately physically written to the SD
>>> card. This slows things done, but is more robust, because there
>>> aren't any buffered changes to lose if power dies, for example. You
>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>> then retry the sketch to see if it changes the speed. If that's the
>>> issue, we should be able to find a way to resolve it.
>>>
>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>
>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>
>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>
>>>> t.
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

2011/2/22 Tom Igoe <>:
> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.

What do you mean by turning it off and on? Is acceptable to have a
way to open the file in buffered or unbuffered mode, with an explicit
sync option for buffered mode? Or do you think we need to be able to
dynamically switch between buffered and unbuffered modes? If the
latter, why? Or maybe we can get away with simply using buffered mode
all the time, and having people sync when needed?

> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.

I did this at one point during development, but it seemed unnecessary
given that you can delete the file before opening it. Would you want
to go back to having FILE_APPEND or FILE_REPLACE instead of
FILE_WRITE? Or distinguish the two some other way?

> t.
>
> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>
>> done!
>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>
>> m
>>
>>
>>
>> 2011/2/11 Tom Igoe <>
>> A possible solution to the slow response from the SD library:
>>
>> t.
>>
>>
>> Begin forwarded message:
>>
>> > From: "David A. Mellis" <>
>> > Date: February 11, 2011 12:11:53 PM EST
>> > To: Tom Igoe <>
>> > Cc: Arduino Developers <>
>> > Subject: Re: [Developers] SD or SPI problem?
>> >
>> > One thing it could be is the fact that we open files in sync mode,
>> > which means all changes are immediately physically written to the SD
>> > card.  This slows things done, but is more robust, because there
>> > aren't any buffered changes to lose if power dies, for example.  You
>> > could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>> > then retry the sketch to see if it changes the speed.  If that's the
>> > issue, we should be able to find a way to resolve it.
>> >
>> > On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>> >> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>> >>
>> >> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>> >> With the SD library:  took 260 seconds to write the same image to the same SD card.
>> >>
>> >> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>> >>
>> >> t.
>> >>
>> >>
>> >> _______________________________________________
>> >> Developers mailing list
>> >>
>> >> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>> >>
>>
>>
>>
>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

That could be very useful. What do you have in mind for the API?

2011/2/22 Limor <>:
> somewhat related, ive been hacking at the library as well to add directory listing and enumeration support. i.e. the problem of how does one "list every file in a dir". its very useful for many different and popular sketch frameworks.
>  i have some code on github but its very much in progress
>
>
> On Feb 22, 2011, at 9:07 PM, Tom Igoe <> wrote:
>
>>    Mustafa and I worked out some new SD card examples today, one that uploads a file to Arduino serially via Processing, and another that downloads a file from Arduino serially. We'll post when they are cleaned up.  But working on both, we ran across two important needs:
>>
>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card.  This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

Which is buffered mode? With O_SYNC on or off? It's much faster with it off, but it does mean you might lose some data if you lose power, right?

I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.

Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?

t.

On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:

> 2011/2/22 Tom Igoe <>:
>> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>
> What do you mean by turning it off and on? Is acceptable to have a
> way to open the file in buffered or unbuffered mode, with an explicit
> sync option for buffered mode? Or do you think we need to be able to
> dynamically switch between buffered and unbuffered modes? If the
> latter, why? Or maybe we can get away with simply using buffered mode
> all the time, and having people sync when needed?
>
>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>
> I did this at one point during development, but it seemed unnecessary
> given that you can delete the file before opening it. Would you want
> to go back to having FILE_APPEND or FILE_REPLACE instead of
> FILE_WRITE? Or distinguish the two some other way?
>
>> t.
>>
>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>
>>> done!
>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>
>>> m
>>>
>>>
>>>
>>> 2011/2/11 Tom Igoe <>
>>> A possible solution to the slow response from the SD library:
>>>
>>> t.
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: "David A. Mellis" <>
>>>> Date: February 11, 2011 12:11:53 PM EST
>>>> To: Tom Igoe <>
>>>> Cc: Arduino Developers <>
>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>
>>>> One thing it could be is the fact that we open files in sync mode,
>>>> which means all changes are immediately physically written to the SD
>>>> card. This slows things done, but is more robust, because there
>>>> aren't any buffered changes to lose if power dies, for example. You
>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>> then retry the sketch to see if it changes the speed. If that's the
>>>> issue, we should be able to find a way to resolve it.
>>>>
>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>
>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>
>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>
> _______________________________________________
> Developers mailing list
>
> http://arduino.cc/mailman/listinfo/developers_arduino.cc


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

By buffered, I mean with O_SYNC off, when writes are buffered in the
memory of the microcontroller before being written to the SD card.
The flush() command will force a sync.

2011/2/23 Tom Igoe <>:
> Which is buffered mode?  With O_SYNC on or off?  It's much faster with it off, but it does mean you might lose some data if you lose power, right?
>
> I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.
>
> Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?
>
> t.
>
> On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:
>
>> 2011/2/22 Tom Igoe <>:
>>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>
>> What do you mean by turning it off and on?  Is acceptable to have a
>> way to open the file in buffered or unbuffered mode, with an explicit
>> sync option for buffered mode?  Or do you think we need to be able to
>> dynamically switch between buffered and unbuffered modes?  If the
>> latter, why?  Or maybe we can get away with simply using buffered mode
>> all the time, and having people sync when needed?
>>
>>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>
>> I did this at one point during development, but it seemed unnecessary
>> given that you can delete the file before opening it.  Would you want
>> to go back to having FILE_APPEND or FILE_REPLACE instead of
>> FILE_WRITE?  Or distinguish the two some other way?
>>
>>> t.
>>>
>>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>>
>>>> done!
>>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>>
>>>> m
>>>>
>>>>
>>>>
>>>> 2011/2/11 Tom Igoe <>
>>>> A possible solution to the slow response from the SD library:
>>>>
>>>> t.
>>>>
>>>>
>>>> Begin forwarded message:
>>>>
>>>>> From: "David A. Mellis" <>
>>>>> Date: February 11, 2011 12:11:53 PM EST
>>>>> To: Tom Igoe <>
>>>>> Cc: Arduino Developers <>
>>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>>
>>>>> One thing it could be is the fact that we open files in sync mode,
>>>>> which means all changes are immediately physically written to the SD
>>>>> card.  This slows things done, but is more robust, because there
>>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>>> issue, we should be able to find a way to resolve it.
>>>>>
>>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>>
>>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>>
>>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>>
>>>>>> t.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Developers mailing list
>>>>>>
>>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>>
>> _______________________________________________
>> Developers mailing list
>>
>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

Oh, perfect, then I think just adding a #define for FILE_WRITE_BUFFERED vs FILE_WRITE will do the trick, and we can write up the differences. OK if i just make that change to the repo?

t.

On Feb 23, 2011, at 8:43 AM, David A. Mellis wrote:

> By buffered, I mean with O_SYNC off, when writes are buffered in the
> memory of the microcontroller before being written to the SD card.
> The flush() command will force a sync.
>
> 2011/2/23 Tom Igoe <>:
>> Which is buffered mode? With O_SYNC on or off? It's much faster with it off, but it does mean you might lose some data if you lose power, right?
>>
>> I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.
>>
>> Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?
>>
>> t.
>>
>> On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:
>>
>>> 2011/2/22 Tom Igoe <>:
>>>> 1) the ability to turn on and off sync. It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>>
>>> What do you mean by turning it off and on? Is acceptable to have a
>>> way to open the file in buffered or unbuffered mode, with an explicit
>>> sync option for buffered mode? Or do you think we need to be able to
>>> dynamically switch between buffered and unbuffered modes? If the
>>> latter, why? Or maybe we can get away with simply using buffered mode
>>> all the time, and having people sync when needed?
>>>
>>>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>>
>>> I did this at one point during development, but it seemed unnecessary
>>> given that you can delete the file before opening it. Would you want
>>> to go back to having FILE_APPEND or FILE_REPLACE instead of
>>> FILE_WRITE? Or distinguish the two some other way?
>>>
>>>> t.
>>>>
>>>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>>>
>>>>> done!
>>>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>>>
>>>>> m
>>>>>
>>>>>
>>>>>
>>>>> 2011/2/11 Tom Igoe <>
>>>>> A possible solution to the slow response from the SD library:
>>>>>
>>>>> t.
>>>>>
>>>>>
>>>>> Begin forwarded message:
>>>>>
>>>>>> From: "David A. Mellis" <>
>>>>>> Date: February 11, 2011 12:11:53 PM EST
>>>>>> To: Tom Igoe <>
>>>>>> Cc: Arduino Developers <>
>>>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>>>
>>>>>> One thing it could be is the fact that we open files in sync mode,
>>>>>> which means all changes are immediately physically written to the SD
>>>>>> card. This slows things done, but is more robust, because there
>>>>>> aren't any buffered changes to lose if power dies, for example. You
>>>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>>>> then retry the sketch to see if it changes the speed. If that's the
>>>>>> issue, we should be able to find a way to resolve it.
>>>>>>
>>>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library. We took the same code, and changed it to the current SD library. Both libraries use sdfatlib under the hood.
>>>>>>>
>>>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>>>> With the SD library: took 260 seconds to write the same image to the same SD card.
>>>>>>>
>>>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation. I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>>>
>>>>>>> t.
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Developers mailing list
>>>>>>>
>>>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>
>>>
>>> _______________________________________________
>>> Developers mailing list
>>>
>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>
>>


_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc

I'll do it.

I think I may just change the default from O_SYNC (non buffered) mode
to non-O_SYNC (buffered) mode. It's not as though we had any
complaints about the buffered mode, it just seemed like a good idea
when the performance implications weren't clear. If your board
randomly crashes or loses power, you'll have problems either way. And
there's still explicit syncing with flush(). I'm not convinced that
the distinction between buffered and unbuffered mode is important
enough in practice to expose in the API.

David

2011/2/23 Tom Igoe <>:
> Oh, perfect,  then I think just adding a #define for FILE_WRITE_BUFFERED vs FILE_WRITE will do the trick, and we can write up the differences.  OK if i just make that change to the repo?
>
> t.
>
> On Feb 23, 2011, at 8:43 AM, David A. Mellis wrote:
>
>> By buffered, I mean with O_SYNC off, when writes are buffered in the
>> memory of the microcontroller before being written to the SD card.
>> The flush() command will force a sync.
>>
>> 2011/2/23 Tom Igoe <>:
>>> Which is buffered mode?  With O_SYNC on or off?  It's much faster with it off, but it does mean you might lose some data if you lose power, right?
>>>
>>> I think if you can choose when you open the file, that would be fine. I think Limor's right that it would be nice to have a sync() command, so you could force a write to the card from time to time, but I don't think it's essential.
>>>
>>> Limor's changes sound interesting too, and we're working on some pages to document the examples here, so perhaps we should take a look at what she's got. LImor, where is it on gitHub, and how in progress is it?
>>>
>>> t.
>>>
>>> On Feb 22, 2011, at 11:25 PM, David A. Mellis wrote:
>>>
>>>> 2011/2/22 Tom Igoe <>:
>>>>> 1) the ability to turn on and off sync.  It seems like that's solved with the O_SYNC change Dave brought up the other day, but we should get that in for the next revision. Turning off sync made the code much more speedy, and reliable.
>>>>
>>>> What do you mean by turning it off and on?  Is acceptable to have a
>>>> way to open the file in buffered or unbuffered mode, with an explicit
>>>> sync option for buffered mode?  Or do you think we need to be able to
>>>> dynamically switch between buffered and unbuffered modes?  If the
>>>> latter, why?  Or maybe we can get away with simply using buffered mode
>>>> all the time, and having people sync when needed?
>>>>
>>>>> 2) the ability to differentiate FILE_WRITE two ways: replacing the file, or appending to it.
>>>>
>>>> I did this at one point during development, but it seemed unnecessary
>>>> given that you can delete the file before opening it.  Would you want
>>>> to go back to having FILE_APPEND or FILE_REPLACE instead of
>>>> FILE_WRITE?  Or distinguish the two some other way?
>>>>
>>>>> t.
>>>>>
>>>>> On Feb 17, 2011, at 6:23 PM, mustafa bağdatlı wrote:
>>>>>
>>>>>> done!
>>>>>> It is even faster dan Sparkfun! Yay arduino! you can see the screen shot fromt he serial and the picture as attached.
>>>>>>
>>>>>> m
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2011/2/11 Tom Igoe <>
>>>>>> A possible solution to the slow response from the SD library:
>>>>>>
>>>>>> t.
>>>>>>
>>>>>>
>>>>>> Begin forwarded message:
>>>>>>
>>>>>>> From: "David A. Mellis" <>
>>>>>>> Date: February 11, 2011 12:11:53 PM EST
>>>>>>> To: Tom Igoe <>
>>>>>>> Cc: Arduino Developers <>
>>>>>>> Subject: Re: [Developers] SD or SPI problem?
>>>>>>>
>>>>>>> One thing it could be is the fact that we open files in sync mode,
>>>>>>> which means all changes are immediately physically written to the SD
>>>>>>> card.  This slows things done, but is more robust, because there
>>>>>>> aren't any buffered changes to lose if power dies, for example.  You
>>>>>>> could try removing O_SYNC from the FILE_WRITE definition in SD.h and
>>>>>>> then retry the sketch to see if it changes the speed.  If that's the
>>>>>>> issue, we should be able to find a way to resolve it.
>>>>>>>
>>>>>>> On Thu, Jan 20, 2011 at 3:30 PM, Tom Igoe <> wrote:
>>>>>>>> We've been playing around with a LinkSprite camera here at ITP, and as part of it one of our researchers ran SparkFun's example that writes to an SD card via the MemoryCard library.  We took the same code, and changed it to the current SD library.  Both libraries use sdfatlib under the hood.
>>>>>>>>
>>>>>>>> With the MemoryCard library: Arduino Uno took approx. 6 seconds to write to an SD card, a 12K file.
>>>>>>>> With the SD library:  took 260 seconds to write the same image to the same SD card.
>>>>>>>>
>>>>>>>> Not sure if the problem is with the SD library, or the SPI library, but it bears some investigation.  I recall we discsussed SPI modes, but can't recall what we defaulted to. Anyone got an idea?
>>>>>>>>
>>>>>>>> t.
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Developers mailing list
>>>>>>>>
>>>>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>>
>>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>>>
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>>
>>>> http://arduino.cc/mailman/listinfo/developers_arduino.cc
>>>
>>>
>
>

_______________________________________________
Developers mailing list

http://arduino.cc/mailman/listinfo/developers_arduino.cc