From matt.brown at kitware.com Sun Mar 10 22:43:23 2019 From: matt.brown at kitware.com (Matt Brown) Date: Sun, 10 Mar 2019 22:43:23 -0400 Subject: [Kwiver-users] Bounding Box Issues Message-ID: I was diagnosing an issue in a detection pipeline doing non-maximal suppression that led me to observe that the area of the intersection of two bounding boxes can be negative. The intersection of two bounding boxes yields a bounding box representing the intersection. Internally, the bboxes are Eigen::AlignedBox, and the intersection uses this Eigen function . Still working on it, but I suspect that Eigen is returning an empty box with junk for parameters (i.e., Eigen says "The result of most methods is undefined when applied to an empty box"). So the question is, what should be returned here when there is no intersection? Should we define a special empty bbox subclass? Best, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.leotta at kitware.com Mon Mar 11 08:55:21 2019 From: matt.leotta at kitware.com (Matt Leotta) Date: Mon, 11 Mar 2019 08:55:21 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: Matt, It sounds like maybe we need to check "isEmpty()" on the box before computing the area, and return 0 for empty boxes. By the way, I'm not a fan of having this bounding box class in KWIVER that simply wraps Eigen::AlignBox, but only exposes a subset of its functions. I think we should just use Eigen::AlignBox and possibly some typedefs for it if we want to call it "bounding_box". Or we could derive a subclass if we need to add some additional methods. There is no reason to hide all the other functionality of AlignBox. This is a separate discussion to have some time. --Matt On Sun, Mar 10, 2019 at 10:43 PM Matt Brown via Kwiver-users < kwiver-users at public.kitware.com> wrote: > I was diagnosing an issue in a detection pipeline doing non-maximal > suppression that led me to observe that the area of the intersection of two > bounding boxes can be negative. > > The intersection > > of two bounding boxes yields a bounding box representing the intersection. > Internally, the bboxes are Eigen::AlignedBox, and the intersection uses this > Eigen function > . > Still working on it, but I suspect that Eigen is returning an empty box > with junk for parameters (i.e., Eigen says "The result of most methods is > undefined when applied to an empty box"). > > So the question is, what should be returned here > > when there is no intersection? Should we define a special empty bbox > subclass? > > Best, > Matt > _______________________________________________ > Kwiver-users mailing list > Kwiver-users at public.kitware.com > https://public.kitware.com/mailman/listinfo/kwiver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Mon Mar 11 10:18:44 2019 From: matt.brown at kitware.com (Matt Brown) Date: Mon, 11 Mar 2019 10:18:44 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: > > It sounds like maybe we need to check "isEmpty()" on the box before > computing the area, and return 0 for empty boxes. > Yeah, I was thinking that would be part of the solution, but every aspect of the bounding box (e.g., top-left/bottom-right coordinates) is invalid when it is empty. When empty, AlignBox still populates those elements but with undefined values, and these get passed on to the Vital bounding_box, allowing the user to query lower_right() and receive undefined values. So, we should either define an empty bounding box subclass, perhaps that returns an error when you ask for lower_right() and forces the programmer to first check isEmpty(), or the intersection function should return a special value when there is no intersection. > By the way, I'm not a fan of having this bounding box class in KWIVER that > simply wraps Eigen::AlignBox, but only exposes a subset of its functions. > I think we should just use Eigen::AlignBox and possibly some typedefs for > it if we want to call it "bounding_box". Or we could derive a subclass if > we need to add some additional methods. There is no reason to hide all the > other functionality of AlignBox. This is a separate discussion to have > some time. > Yes, I see your point. Would like to have this discussion some time in the near future. Best, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From arslan.basharat at kitware.com Mon Mar 11 10:26:24 2019 From: arslan.basharat at kitware.com (Arslan Basharat) Date: Mon, 11 Mar 2019 10:26:24 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: intersection() could throw a warning that one of the boxes is empty, but it should still return 0 intersection area, I would think. Is there a reason to utilize bounding_box class to also support boxes that are not axis aligned, which is all Eigen::AlignBox supports? -Arslan On Mon, Mar 11, 2019 at 10:19 AM Matt Brown via Kwiver-users < kwiver-users at public.kitware.com> wrote: > It sounds like maybe we need to check "isEmpty()" on the box before >> computing the area, and return 0 for empty boxes. >> > Yeah, I was thinking that would be part of the solution, but every aspect > of the bounding box (e.g., top-left/bottom-right coordinates) is invalid > when it is empty. When empty, AlignBox still populates those elements but > with undefined values, and these get passed on to the Vital bounding_box, > allowing the user to query lower_right() and receive undefined values. So, > we should either define an empty bounding box subclass, perhaps that > returns an error when you ask for lower_right() and forces the programmer > to first check isEmpty(), or the intersection function should return a > special value when there is no intersection. > > >> By the way, I'm not a fan of having this bounding box class in KWIVER >> that simply wraps Eigen::AlignBox, but only exposes a subset of its >> functions. I think we should just use Eigen::AlignBox and possibly some >> typedefs for it if we want to call it "bounding_box". Or we could derive a >> subclass if we need to add some additional methods. There is no reason to >> hide all the other functionality of AlignBox. This is a separate >> discussion to have some time. >> > Yes, I see your point. Would like to have this discussion some time in the > near future. > > Best, > Matt > _______________________________________________ > Kwiver-users mailing list > Kwiver-users at public.kitware.com > https://public.kitware.com/mailman/listinfo/kwiver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.tunison at kitware.com Mon Mar 11 10:39:29 2019 From: paul.tunison at kitware.com (Paul Tunison) Date: Mon, 11 Mar 2019 10:39:29 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: On Mon, Mar 11, 2019 at 10:27 AM Arslan Basharat via Kwiver-users < kwiver-users at public.kitware.com> wrote: > intersection() could throw a warning that one of the boxes is empty, but > it should still return 0 intersection area, I would think. > > Is there a reason to utilize bounding_box class to also support boxes that > are not axis aligned, which is all Eigen::AlignBox supports? > Good point. We've had issues before with prematurely locking things to integers (may have been bounding boxes in vidtk, actually, maybe something else like image coordinates?). Unless the purpose of the bounding box concept in this case is a purposeful optimization we may want to have floating-point bounding boxes otherwise stacked transformations will also have stacked integer-casting errors. > > -Arslan > > On Mon, Mar 11, 2019 at 10:19 AM Matt Brown via Kwiver-users < > kwiver-users at public.kitware.com> wrote: > >> It sounds like maybe we need to check "isEmpty()" on the box before >>> computing the area, and return 0 for empty boxes. >>> >> Yeah, I was thinking that would be part of the solution, but every aspect >> of the bounding box (e.g., top-left/bottom-right coordinates) is invalid >> when it is empty. When empty, AlignBox still populates those elements but >> with undefined values, and these get passed on to the Vital bounding_box, >> allowing the user to query lower_right() and receive undefined values. So, >> we should either define an empty bounding box subclass, perhaps that >> returns an error when you ask for lower_right() and forces the programmer >> to first check isEmpty(), or the intersection function should return a >> special value when there is no intersection. >> >> >>> By the way, I'm not a fan of having this bounding box class in KWIVER >>> that simply wraps Eigen::AlignBox, but only exposes a subset of its >>> functions. I think we should just use Eigen::AlignBox and possibly some >>> typedefs for it if we want to call it "bounding_box". Or we could derive a >>> subclass if we need to add some additional methods. There is no reason to >>> hide all the other functionality of AlignBox. This is a separate >>> discussion to have some time. >>> >> Yes, I see your point. Would like to have this discussion some time in >> the near future. >> >> Best, >> Matt >> _______________________________________________ >> Kwiver-users mailing list >> Kwiver-users at public.kitware.com >> https://public.kitware.com/mailman/listinfo/kwiver-users >> > _______________________________________________ > Kwiver-users mailing list > Kwiver-users at public.kitware.com > https://public.kitware.com/mailman/listinfo/kwiver-users > -- Paul Tunison Senior R&D Engineer, Computer Vision Kitware, Inc. 1712 Route 9, Suite 300 Clifton Park, NY 12065 USA Phone: (518) 371-3971 Ext.1164 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Mon Mar 11 10:46:03 2019 From: matt.brown at kitware.com (Matt Brown) Date: Mon, 11 Mar 2019 10:46:03 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: > > intersection() could throw a warning that one of the boxes is empty, but > it should still return 0 intersection area, I would think. > The issue is, intersection() returns a bounding box, not an area. So there isn't a clear ideal way to return a bounding box that properly encodes "empty." Right now, it returns a bounding box with all properties populated in an undefined manner. > Is there a reason to utilize bounding_box class to also support boxes that > are not axis aligned, which is all Eigen::AlignBox supports? > I think non-axis-aligned bounding boxes might have utility in the future, but bounding_box is currently configured to only specify axis-aligned boxes. So, we would have to completely redefine the interface to bounding_box to support non-axis-aligned boxes. ~Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Mon Mar 11 10:48:13 2019 From: matt.brown at kitware.com (Matt Brown) Date: Mon, 11 Mar 2019 10:48:13 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: > > Good point. We've had issues before with prematurely locking things to > integers (may have been bounding boxes in vidtk, actually, maybe something > else like image coordinates?). Unless the purpose of the bounding box > concept in this case is a purposeful optimization we may want to have > floating-point bounding boxes otherwise stacked transformations will also > have stacked integer-casting errors. > bounding_box appears to be templated over coordinate type (int, float, double, ...) ~Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.tunison at kitware.com Mon Mar 11 10:51:24 2019 From: paul.tunison at kitware.com (Paul Tunison) Date: Mon, 11 Mar 2019 10:51:24 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: Yup, you're right, I misunderstood the words used. On Mon, Mar 11, 2019 at 10:48 AM Matt Brown wrote: > Good point. We've had issues before with prematurely locking things to >> integers (may have been bounding boxes in vidtk, actually, maybe something >> else like image coordinates?). Unless the purpose of the bounding box >> concept in this case is a purposeful optimization we may want to have >> floating-point bounding boxes otherwise stacked transformations will also >> have stacked integer-casting errors. >> > bounding_box appears to be templated over coordinate type (int, float, > double, ...) > > ~Matt > -- Paul Tunison Senior R&D Engineer, Computer Vision Kitware, Inc. 1712 Route 9, Suite 300 Clifton Park, NY 12065 USA Phone: (518) 371-3971 Ext.1164 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.leotta at kitware.com Mon Mar 11 10:55:28 2019 From: matt.leotta at kitware.com (Matt Leotta) Date: Mon, 11 Mar 2019 10:55:28 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: I'm not sure what you mean by "properly encodes empty". The box does encode empty by having at least one max point less then its min point. There is a function isEmpty() that checks if a box is empty. I've used bound box classes in several libraries that all work this way. It is the user's responsibility to check if a box is empty (call isEmpty()) before doing any operations that require a valid box. I'm not sure there is a better approach here, except maybe that the area/volume of an empty box should return 0. On Mon, Mar 11, 2019 at 10:46 AM Matt Brown wrote: > intersection() could throw a warning that one of the boxes is empty, but >> it should still return 0 intersection area, I would think. >> > The issue is, intersection() returns a bounding box, not an area. So there > isn't a clear ideal way to return a bounding box that properly encodes > "empty." Right now, it returns a bounding box with all properties populated > in an undefined manner. > > >> Is there a reason to utilize bounding_box class to also support boxes >> that are not axis aligned, which is all Eigen::AlignBox supports? >> > I think non-axis-aligned bounding boxes might have utility in the future, > but bounding_box is currently configured to only specify axis-aligned > boxes. So, we would have to completely redefine the interface to > bounding_box to support non-axis-aligned boxes. > > ~Matt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roddy.collins at kitware.com Mon Mar 11 10:57:37 2019 From: roddy.collins at kitware.com (Roddy Collins) Date: Mon, 11 Mar 2019 10:57:37 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: On Mon, Mar 11, 2019 at 10:46 AM Matt Brown via Kwiver-users < kwiver-users at public.kitware.com> wrote: > intersection() could throw a warning that one of the boxes is empty, but >> it should still return 0 intersection area, I would think. >> > The issue is, intersection() returns a bounding box, not an area. So there > isn't a clear ideal way to return a bounding box that properly encodes > "empty." Right now, it returns a bounding box with all properties populated > in an undefined manner. > vgl_intersection returns an empty box, which is always min(x,y)=1, max(x,y)=0; vgl_box_2d's implementation of is_empty() returns true whenever the min coordinates are less than the max. Given how often vgl_intersection appears in our legacy code (probably other vgl code as well), would it make sense to copy over the tests from vgl to make sure the semantics are the same? -Roddy -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Mon Mar 11 11:33:31 2019 From: matt.brown at kitware.com (Matt Brown) Date: Mon, 11 Mar 2019 11:33:31 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: > > I'm not sure what you mean by "properly encodes empty". The box does > encode empty by having at least one max point less then its min point. > There is a function isEmpty() that checks if a box is empty. I've used > bound box classes in several libraries that all work this way. It is the > user's responsibility to check if a box is empty (call isEmpty()) before > doing any operations that require a valid box. I'm not sure there is a > better approach here, except maybe that the area/volume of an empty box > should return 0. > Ok, I suppose that makes sense. I'd say we should at least: - Return zero for area/volume of an empty box. - Add sufficient documentation to bounding_box to make it clear that inverting upper/left and lower/right represents "empty" because that wasn't particularly clear to me at the start. - Add an is_empty method to bounding_box because it is more efficient to set this at creation versus having someone explicitly check the different combos of min > max. Optionally, - Does it also make sense to raise an error if you try to access upper_left() or lower_right() for an empty box as I can't think of a use case where you would want to know what that values are there other than to check is_empty(). If someone wants to warp bounding boxes to another coordinate system that may have the axes inverted, do we assume it is the implementer's responsibility to know that inverting an axis will make the bounding box represent empty? Best, Matt > > > On Mon, Mar 11, 2019 at 10:46 AM Matt Brown > wrote: > >> intersection() could throw a warning that one of the boxes is empty, but >>> it should still return 0 intersection area, I would think. >>> >> The issue is, intersection() returns a bounding box, not an area. So >> there isn't a clear ideal way to return a bounding box that properly >> encodes "empty." Right now, it returns a bounding box with all properties >> populated in an undefined manner. >> >> >>> Is there a reason to utilize bounding_box class to also support boxes >>> that are not axis aligned, which is all Eigen::AlignBox supports? >>> >> I think non-axis-aligned bounding boxes might have utility in the future, >> but bounding_box is currently configured to only specify axis-aligned >> boxes. So, we would have to completely redefine the interface to >> bounding_box to support non-axis-aligned boxes. >> >> ~Matt >> > -- Matt Brown, Ph.D. Staff R&D Engineer Kitware, Inc. 101 E Weaver St. Suite G4 Carrboro, NC 27510 (908) 910-3385 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.leotta at kitware.com Mon Mar 11 11:51:11 2019 From: matt.leotta at kitware.com (Matt Leotta) Date: Mon, 11 Mar 2019 11:51:11 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: On Mon, Mar 11, 2019 at 11:33 AM Matt Brown wrote: > I'm not sure what you mean by "properly encodes empty". The box does >> encode empty by having at least one max point less then its min point. >> There is a function isEmpty() that checks if a box is empty. I've used >> bound box classes in several libraries that all work this way. It is the >> user's responsibility to check if a box is empty (call isEmpty()) before >> doing any operations that require a valid box. I'm not sure there is a >> better approach here, except maybe that the area/volume of an empty box >> should return 0. >> > > Ok, I suppose that makes sense. I'd say we should at least: > > - Return zero for area/volume of an empty box. > > Yes, probably, but it would be good to understand why Eigen doesn't do this. It might just be for efficiency. > > - Add sufficient documentation to bounding_box to make it clear that > inverting upper/left and lower/right represents "empty" because that wasn't > particularly clear to me at the start. > > If we just use the Eigen class directly we can refer to the existing documentation for that class > > - Add an is_empty method to bounding_box because it is more efficient > to set this at creation versus having someone explicitly check the > different combos of min > max. > > Or just use the Eigen class directly so we don't have to keep exposing more underlying functions. > Optionally, > > - Does it also make sense to raise an error if you try to access > upper_left() or lower_right() for an empty box as I can't think of a use > case where you would want to know what that values are there other than to > check is_empty(). > > Probably not. This should be simple, fast accessors. We probably don't want to overhead of checking is_empty for each access. It is the user's responsibility to know if the box is empty. > If someone wants to warp bounding boxes to another coordinate system that > may have the axes inverted, do we assume it is the implementer's > responsibility to know that inverting an axis will make the bounding box > represent empty? > You should not hit this case. This is a bounding box, not a polygon. You should warp the corners of this box (or other data within) and then compute a new bounding box on the warped points. > > Best, > Matt > > >> >> >> On Mon, Mar 11, 2019 at 10:46 AM Matt Brown >> wrote: >> >>> intersection() could throw a warning that one of the boxes is empty, but >>>> it should still return 0 intersection area, I would think. >>>> >>> The issue is, intersection() returns a bounding box, not an area. So >>> there isn't a clear ideal way to return a bounding box that properly >>> encodes "empty." Right now, it returns a bounding box with all properties >>> populated in an undefined manner. >>> >>> >>>> Is there a reason to utilize bounding_box class to also support boxes >>>> that are not axis aligned, which is all Eigen::AlignBox supports? >>>> >>> I think non-axis-aligned bounding boxes might have utility in the >>> future, but bounding_box is currently configured to only specify >>> axis-aligned boxes. So, we would have to completely redefine the interface >>> to bounding_box to support non-axis-aligned boxes. >>> >>> ~Matt >>> >> > > -- > Matt Brown, Ph.D. > Staff R&D Engineer > Kitware, Inc. > 101 E Weaver St. Suite G4 > Carrboro, NC 27510 > (908) 910-3385 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Mon Mar 11 11:59:46 2019 From: matt.brown at kitware.com (Matt Brown) Date: Mon, 11 Mar 2019 11:59:46 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: > > You should not hit this case. This is a bounding box, not a polygon. You > should warp the corners of this box (or other data within) and then compute > a new bounding box on the warped points. > Maybe this is contrived, but as an example, let's say you are working with orbiting WAMI data and you warp a raw-image bbox from one side of the orbit to a raw image from the other side of the orbit. The top/left and bottom/right corners may get inverted so that top/left > bottom/right. If you naively generated a new bbox with that top/left and bottom/right specification, the new bbox encodes empty. Is it the detection-warping-code implementer's responsibility to manage this accordingly? Best, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.leotta at kitware.com Mon Mar 11 12:10:10 2019 From: matt.leotta at kitware.com (Matt Leotta) Date: Mon, 11 Mar 2019 12:10:10 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: On Mon, Mar 11, 2019 at 11:59 AM Matt Brown wrote: > You should not hit this case. This is a bounding box, not a polygon. You >> should warp the corners of this box (or other data within) and then compute >> a new bounding box on the warped points. >> > Maybe this is contrived, but as an example, let's say you are working with > orbiting WAMI data and you warp a raw-image bbox from one side of the orbit > to a raw image from the other side of the orbit. The top/left and > bottom/right corners may get inverted so that top/left > bottom/right. If > you naively generated a new bbox with that top/left and bottom/right > specification, the new bbox encodes empty. Is it the detection-warping-code > implementer's responsibility to manage this accordingly? > > The right way to do this is to warp all four corner points and then add them to a new empty box with the "extend()" function to recompute the new box. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Mon Mar 11 12:55:38 2019 From: matt.brown at kitware.com (Matt Brown) Date: Mon, 11 Mar 2019 12:55:38 -0400 Subject: [Kwiver-users] Bounding Box Issues In-Reply-To: References: Message-ID: Until we refactor according to Matt L.'s suggested changes, one remaining problem is that in cases where two bounding boxes don't intersect, intersect() can still return a bounding box that has bottom-right > top-left, indicating it is a non-empty box, but those top-left and bottom-right coordinates are garbage values that you would need to look into Eigen::AlignedBox and query isEmpty() to know whether it is valid. Presumably the solution in intersection() is to check isEmpty() on the Eigen::AlignedBox and if it is, generate a bounding box with top-left > bottom-right. That is why it might be convenient to define a function generate an empty bounding_box, perhaps compliant with Roddy's specified vgl_intersection convention for an empty box. On Mon, Mar 11, 2019 at 12:11 PM Matt Leotta wrote: > > > On Mon, Mar 11, 2019 at 11:59 AM Matt Brown > wrote: > >> You should not hit this case. This is a bounding box, not a polygon. >>> You should warp the corners of this box (or other data within) and then >>> compute a new bounding box on the warped points. >>> >> Maybe this is contrived, but as an example, let's say you are working >> with orbiting WAMI data and you warp a raw-image bbox from one side of the >> orbit to a raw image from the other side of the orbit. The top/left and >> bottom/right corners may get inverted so that top/left > bottom/right. If >> you naively generated a new bbox with that top/left and bottom/right >> specification, the new bbox encodes empty. Is it the detection-warping-code >> implementer's responsibility to manage this accordingly? >> >> The right way to do this is to warp all four corner points and then add > them to a new empty box with the "extend()" function to recompute the new > box. > -- Matt Brown, Ph.D. Staff R&D Engineer Kitware, Inc. 101 E Weaver St. Suite G4 Carrboro, NC 27510 (908) 910-3385 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Tue Mar 12 16:17:43 2019 From: matt.brown at kitware.com (Matt Brown) Date: Tue, 12 Mar 2019 16:17:43 -0400 Subject: [Kwiver-users] Detected object set sorted by confidence Message-ID: I notice that the documentation for *detected_object_set* says "detections are ordered on their basic confidence value." I'm not quite sure what is meant by "basic confidence value" or the full implications for cases where a detection has a confidence for multiple classes. An example where this becomes a problem is my current debugging of a non-maximum suppression filter, which tries to identify overlapping detections and only take the highest-confidence one. The current non-max filter assumes the detections in *detected_object_set* are sorted by confidence, and as it loops over the set, if there is bounding box overlap, the detection earlier in the set wins. The problem is, the *detected_object_set *will at most be sorted on one class's confidence, and I believe there are several examples already where other algorithms add or merge sets without enforcing that they remain sorted. In fact, the add function for *detected_object_set* itself just appends to the end of the vector. My suggestion is that we remove the specification that *detected_object_set* is sorted by confidence. This makes sense, at least nominally, since a set implies an unordered group (versus a list, tuple, vector, etc.). Otherwise, I think we'd need a way to ask *detected_object_set* if it is sorted on a particular class, Best, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From linus.sherrill at kitware.com Tue Mar 12 16:34:11 2019 From: linus.sherrill at kitware.com (Linus Sherrill) Date: Tue, 12 Mar 2019 16:34:11 -0400 Subject: [Kwiver-users] Detected object set sorted by confidence In-Reply-To: References: Message-ID: You are correct, hat the comment is no longer correct and should be removed. There was a time when the elements were sorted by default. Now the elements can be retrieved in an ordered fashion using the select() methods and are only sorted on request. -Linus On Tue, Mar 12, 2019 at 4:18 PM Matt Brown via Kwiver-users < kwiver-users at public.kitware.com> wrote: > I notice that the documentation for *detected_object_set* says "detections > are ordered on their basic confidence value." > > I'm not quite sure what is meant by "basic confidence value" or the full > implications for cases where a detection has a confidence for multiple > classes. > > An example where this becomes a problem is my current debugging of a > non-maximum suppression filter, which tries to identify overlapping > detections and only take the highest-confidence one. The current non-max > filter assumes the detections in *detected_object_set* are sorted by > confidence, and as it loops over the set, if there is bounding box overlap, > the detection earlier in the set wins. The problem is, the *detected_object_set > *will at most be sorted on one class's confidence, and I believe there > are several examples already where other algorithms add or merge sets > without enforcing that they remain sorted. In fact, the add function > > for *detected_object_set* itself just appends to the end of the vector. > > My suggestion is that we remove the specification that > *detected_object_set* is sorted by confidence. This makes sense, at least > nominally, since a set implies an unordered group (versus a list, tuple, > vector, etc.). Otherwise, I think we'd need a way to ask > *detected_object_set* if it is sorted on a particular class, > > Best, > Matt > _______________________________________________ > Kwiver-users mailing list > Kwiver-users at public.kitware.com > https://public.kitware.com/mailman/listinfo/kwiver-users > -- *Linus Sherrill* Facility Security Officer Kitware, Inc. 1712 Route 9, Suite 300 Clifton Park, NY 12065 E: linus.sherrill at kitware.com P: 518.881.4400 The information transmitted is intended only for the person or entity to which it is addressed and may contain privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information, by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and destroy the e-mail. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.brown at kitware.com Wed Mar 13 12:38:50 2019 From: matt.brown at kitware.com (Matt Brown) Date: Wed, 13 Mar 2019 12:38:50 -0400 Subject: [Kwiver-users] detected_object confidence Message-ID: This is a follow-on from my recent email, but I think it is a new discussion. A *detected_object* has a confidence associated with it, but it also carries a *detected_object_type *, which lets you specify scores for a number of different classification types. As of now, neither documentation nor implementation enforces any correlation between the confidence and the individual class scores. It seems one of the major use cases for confidence is sorting a *detected_object_set* based on each *detected_object.confidence*, but this is very confusing in cases where there are multiple classes associated with the *detect_object_type*. Detectors with an associated classifier for many classes is becoming more and more common for us. For example, we might have a faster-rcnn detector that returns a score for *background*, *person*, *car*, *person-holding-gun*, etc., and in which case, it isn't clear what the most-useful specification of the overall detection's confidence would be. I think the current state of things is a recipe for mistakes and confusion. One option is to just decide on a convention and document it thoroughly. Alternatively, we could just remove reference to confidence and require looking into the detected_object_type scores. Though, I'm not sure I understand all the use cases that this would break. Best, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: