#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Why?

Should ctx still be static anyway?

Should it be static in every rt_pg function?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Why?

Should ctx still be static anyway?

Should it be static in every rt_pg function?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Did you get any interesing traces about your crash yet ?

I was just guessing that the memory context in which ctx is allocated is
being cleared while still used (in the static case). Using malloc for
allocating it would confirm that's the case.

I don't have much time atm to take a closer look at the issue.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Why?

Should ctx still be static anyway?

Should it be static in every rt_pg function?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Did you get any interesing traces about your crash yet ?

I was just guessing that the memory context in which ctx is allocated is
being cleared while still used (in the static case). Using malloc for
allocating it would confirm that's the case.

I don't have much time atm to take a closer look at the issue.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe.

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Why?

Should ctx still be static anyway?

Should it be static in every rt_pg function?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Did you get any interesing traces about your crash yet ?

I was just guessing that the memory context in which ctx is allocated is
being cleared while still used (in the static case). Using malloc for
allocating it would confirm that's the case.

I don't have much time atm to take a closer look at the issue.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
Changes (by jorgearevalo):

* status: new => assigned


Comment:

For the record: After 8h executing the query in a 64bits machine, with a
10x10px tiled raster, it took 240MB. 180MB from heap. And growing. As
Pierre said, typical memory leak.

Same symptom in MapAlgebra, like Pierre said too. And there're 2 common
potential memory-leak points:[[BR]]
- rt_context_new: context allocated is not freed. It should be always
freed.[[BR]]
- rt_raster_deserialize: allocated memory for raster object. It should be
always freed. Even when the function returns that raster, because the
raster is serialized again before returning it.[[BR]]

Freeing that memory should solve the memory leak. Testing it.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Why?

Should ctx still be static anyway?

Should it be static in every rt_pg function?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Did you get any interesing traces about your crash yet ?

I was just guessing that the memory context in which ctx is allocated is
being cleared while still used (in the static case). Using malloc for
allocating it would confirm that's the case.

I don't have much time atm to take a closer look at the issue.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
Changes (by jorgearevalo):

* status: new => assigned


Comment:

For the record: After 8h executing the query in a 64bits machine, with a
10x10px tiled raster, it took 240MB. 180MB from heap. And growing. As
Pierre said, typical memory leak.

Same symptom in MapAlgebra, like Pierre said too. And there're 2 common
potential memory-leak points:[[BR]]
- rt_context_new: context allocated is not freed. It should be always
freed.[[BR]]
- rt_raster_deserialize: allocated memory for raster object. It should be
always freed. Even when the function returns that raster, because the
raster is serialized again before returning it.[[BR]]

Freeing that memory should solve the memory leak. Testing it.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Note that the raster object memory should be already freed by postgresql
itself,
assuming the memory context is correct. Doesn't hurt to be explicit about
that though.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Why?

Should ctx still be static anyway?

Should it be static in every rt_pg function?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Did you get any interesing traces about your crash yet ?

I was just guessing that the memory context in which ctx is allocated is
being cleared while still used (in the static case). Using malloc for
allocating it would confirm that's the case.

I don't have much time atm to take a closer look at the issue.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
Changes (by jorgearevalo):

* status: new => assigned


Comment:

For the record: After 8h executing the query in a 64bits machine, with a
10x10px tiled raster, it took 240MB. 180MB from heap. And growing. As
Pierre said, typical memory leak.

Same symptom in MapAlgebra, like Pierre said too. And there're 2 common
potential memory-leak points:[[BR]]
- rt_context_new: context allocated is not freed. It should be always
freed.[[BR]]
- rt_raster_deserialize: allocated memory for raster object. It should be
always freed. Even when the function returns that raster, because the
raster is serialized again before returning it.[[BR]]

Freeing that memory should solve the memory leak. Testing it.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Note that the raster object memory should be already freed by postgresql
itself,
assuming the memory context is correct. Doesn't hurt to be explicit about
that though.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by jorgearevalo):

Yes, the pgraster object is freed by postgresql, but the raster object
deserialized with a core function should be explicitly freed, like the
context created. Correct me if I'm wrong.

Added some memory deallocations in r7008, but a small memory grow still
exist. Debugging.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)

#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
A query like this:

SELECT ST_Value(rast, 1, x, y)
FROM generate_series(1, 10) x, generate_series(1, 10) y,
srtm_22_03_tiled_10x10

causes the memory to grow indefinitely finally returning an error. There
seems to be a memory leek somewhere. I am wondering if we should not use
PG_FREE_IF_COPY(pgraster,0) in rt_pg.c

Also: Smaller the tiles, faster the memory grows. But I think this is only
related to the page size. Equivalent area splitted in small tiles just
loads faster.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

The problem seems to come from "get_rt_context" which switch the context
of the most PostGIS raster functions preventing the PostgreSQL function
manager to free the memory allocated to the context.

Shall we not delete this context using rt_context_destroy(ctx) in all the
functions except those returning PG_RETURN_POINTER(pgraster)?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

I've the impression that the ctx variable within get_rt_context was meant
to be a static.
Or the if ( ctx ) return ctx; would make no sense.

Haven't tried it myself, but would be worth a quick test.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Making it static in the get_rt_context cause the server to crash. Should
it be static in every function or rt_pg?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Uhm.. the TODO in the function might be the cause of the crash (destroyed
too early by
postgresql ?)
Where does the crash happen ? Have a backtrace or even better a valgrind
report?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

I know that if I don't make it static and I use
rt_context_destroy(rt_context ctx) at the end of the proper rt_pg
functions, everything goes fine and momory do not grow anymore. My guess
is that this recipe should not be applied to function returning
PG_RETURN_POINTER(pgraster)

But making it static seems to be nicer, so I will trace a little bit.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Fixing rt_context_new itself to use malloc rather than the provided
allocator might be a quicker fix than playing with the memory context
switch.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by pracine):

Why?

Should ctx still be static anyway?

Should it be static in every rt_pg function?

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: pracine
Type: defect | Status: new
Priority: critical | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Did you get any interesing traces about your crash yet ?

I was just guessing that the memory context in which ctx is allocated is
being cleared while still used (in the static case). Using malloc for
allocating it would confirm that's the case.

I don't have much time atm to take a closer look at the issue.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
___________________________________________________

Posted on the PostGIS : postgis-devel mailing list. Go to http://postgis.refractions.net/mailman/listinfo/postgis-devel to subscribe. #851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------
Changes (by jorgearevalo):

* status: new => assigned


Comment:

For the record: After 8h executing the query in a 64bits machine, with a
10x10px tiled raster, it took 240MB. 180MB from heap. And growing. As
Pierre said, typical memory leak.

Same symptom in MapAlgebra, like Pierre said too. And there're 2 common
potential memory-leak points:[[BR]]
- rt_context_new: context allocated is not freed. It should be always
freed.[[BR]]
- rt_raster_deserialize: allocated memory for raster object. It should be
always freed. Even when the function returns that raster, because the
raster is serialized again before returning it.[[BR]]

Freeing that memory should solve the memory leak. Testing it.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

Note that the raster object memory should be already freed by postgresql
itself,
assuming the memory context is correct. Doesn't hurt to be explicit about
that though.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by jorgearevalo):

Yes, the pgraster object is freed by postgresql, but the raster object
deserialized with a core function should be explicitly freed, like the
context created. Correct me if I'm wrong.

Added some memory deallocations in r7008, but a small memory grow still
exist. Debugging.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)
#851: [raster] Numerous ST_Value query cause the memory to grow indefinitely
----------------------------+-----------------------------------------------
Reporter: pracine | Owner: jorgearevalo
Type: defect | Status: assigned
Priority: blocker | Milestone: PostGIS 2.0.0
Component: postgis raster | Version: trunk
Keywords: |
----------------------------+-----------------------------------------------

Comment(by strk):

The core deserializer uses allocators set in the rt_context.
In the rt_pg case, this is palloc, which is supposed to release the memory
when the associated pool goes out of scope.

Now, the scope might be wider than you'd like, in which case it's fine to
release manually, but it shouldn't live cross a single query.

--
Ticket URL:
PostGIS
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
_______________________________________________
postgis-devel mailing list
postgis-
http://postgis.refractions.net/mailman/listinfo/postgis-devel
)