[pygr-notify] Issue 133 in pygr: libnclist: find_suboverlap_start doesn't check isub against nlists, can segfault

[pygr at googlecode.com]

Status: New
Owner: pygr-bug... at googlegroups.com
Labels: Type-Defect Priority-Medium

New issue 133 by Deewi... at gmail.com: libnclist: find_suboverlap_start  
doesn't check isub against nlists, can segfault
http://code.google.com/p/pygr/issues/detail?id=133

Using libnclist from pygr 0.8.2, the find_suboverlap_start function used by  
find_intervals can cause a segfault. This can happen at least when there  
are no SublistHeaders, maybe otherwise as well; I'm not sure. The following  
simple C code demonstrates the issue:

=====
#define BUILD_C_LIBRARY
#include "intervaldb.h"
int main() {
	const int n = 4;
	IntervalMap im[n];
	unsigned i;

	for (i = 0; i < 4; ++i) {
		im[i].start = i;
		im[i].end = 2*(i+1);
		im[i].target_id = 0;
	}

	int ntop, nlists;
	SublistHeader *subheader = build_nested_list_inplace(im, 4, &ntop,  
&nlists);

	IntervalIterator *ito = interval_iterator_alloc();

	IntervalMap buf[1];
	for (IntervalIterator *it = ito; it;) {
		int found;
		find_intervals(it, 0, 8, im, n, subheader, nlists, buf, 1, &found, &it);
		if (found == 1)
			printf("Found overlap: %d %d\n", buf[0].start, buf[0].end);
	}

	free_interval_iterator(ito);
}
=====

When run, this code segfaults, since nlists is 0 and thus subheader is an  
empty array and find_overlap_start accesses the invalid pointer  
subheader[0].

To fix, add a check that "isub < nlists" in the first if statement in  
find_overlap_start.