Ruby not returning correct dimensions  Topic is solved

Ruby not returning correct dimensions

Postby Frankn » Fri Jul 14, 2017 6:04 pm

Sorry in advance for what may be a really noob question and I know I've seen this before but been away from scripting for a while and can't remember and can't find the solution by searching...

Basically if I have 3 components in my model each being 1/2" thick when I retrieve the components dimensions in ruby, 2 of them return 0.5 and one returns 0.4999999999999716. Why? and how do my fix that? I did try adding .to_l in my code but that didn't help.

Thanks
0

Frankn 
 

Re: Ruby not returning correct dimensions  Topic is solved

Postby TIG » Fri Jul 14, 2017 6:29 pm

0.4999999999999716" IS for all intents and purposes 0.5"


All computing code will have some approximation built in to it.
SketchUp's is 1/1000"
That's 0.001"
So quite clearly 0.4999999999999716" is equivalent to 0.5" for all practical purposes.

If you need to output the value and you are unhappy with the 'nearness' there are code method to return a user expected 'dim value'...
0
TIG
User avatar
TIG 
Global Moderator
 

Re: Ruby not returning correct dimensions

Postby Frankn » Mon Jul 17, 2017 11:46 pm

Thanks for the reply and clarification Tig!

The issue I'm having is that I'm using group_by to group by thickness but since the values don't match group_by doesn't work. So knowing what you just explained I'm using 0.4999999999999716.round(6) and that seems to do the trick. I use (6) because some values in inches have six digits after the .

If there's another way fix this issue please let me know.
0

Frankn 
 

Re: Ruby not returning correct dimensions

Postby TIG » Tue Jul 18, 2017 10:15 am

You have found your own solution.
;)
0
TIG
User avatar
TIG 
Global Moderator
 

SketchUcation One-Liner Adverts

by Ad Machine » 5 minutes ago



Ad Machine 
Robot
 



 

Return to Developers' Forum

Who is online

Users browsing this forum: Bing [Bot], darkphas, DotSketch and 3 guests

Visit our sponsors: