javascript - Why is 10000000000000.126.toString() 1000000000000.127 (and what can I do to prevent it)? -


why 10000000000000.126.tostring() 1000000000000.127 , 100000000000.126.tostring() not?

i think must have maximum value of number in js (as per this question), related floating point operations , how?

i'm asking because wrote function format number using thousands separators , prevent this.

function th(n,sep) {     sep = sep || '.';     var dec = n.tostring().split(/[,.]/),         narr = dec[0].split(''),         isdot = /\./.test(sep);     return function tt(n) {               return n.length > 3 ?                tt(n.slice(0,n.length-3)).concat(n.slice(n.length-3).join('')) :                [n.join('')]             ;         }(narr)         .join(sep)         + (dec[1] ? (isdot?',':'.') + dec[1] : ''); } sep1000(10000000000000.126); //=> 10.000.000.000.000,127 sep1000(1000000000000.126); //=> 1.000.000.000.000,126

because not numbers can represented floating point (javascript uses double-precision 64-bit format ieee 754 numbers), rounding errors come in. instance:

alert(0.1 + 0.2); // "0.30000000000000004"

all numbering systems limited storage (e.g., numbering systems) have issue, , used dealing our decimal system (which can't accurately represent "one third") , surprised of different values floating-point formats used computers can't accurately represent. sort of thing why you're seeing more , more "decimal" style types out there (java has bigdecimal, c# has decimal, etc.), use our style of number representation (at cost) , useful applications rounding needs align our expectations more closely (such financial apps).

update: haven't tried, may able work around manipulating values bit before grab strings. instance, works specific example (live copy):

code:

function display(msg) {   var p = document.createelement('p');   p.innerhtml = msg;   document.body.appendchild(p); }  function precisetostring(num) {   var floored = math.floor(num),       fraction = num - floored,       rv,       fractionstring,       n;    rv = string(floored);   n = rv.indexof(".");   if (n >= 0) {     rv = rv.substring(0, n);   }   fractionstring = string(fraction);   if (fractionstring.substring(0, 2) !== "0.") {      return string(num); // punt   }   rv += "." + fractionstring.substring(2);   return rv; }  display(precisetostring(10000000000000.126));

result:

10000000000000.126953125

...which can, of course, truncated see fit. of course, it's important note 10000000000000.126953125 != 10000000000000.126. think ship had sailed (e.g., number contained imprecise value), given seeing .127. can't see way know original went 3 places, not number.

i'm not saying above in way reliable, you'd have put through paces prove it's (which say, i'm) not doing stoopid there. , again, since don't know precision ended in first place, i'm not sure how helpful is.


Comments

Post a Comment

Popular posts from this blog

c++ - Is it possible to compile a VST on linux? -

for class project i'm attempting write vst plugin backed cuda. current cuda workflow on linux box, i'd prefer compile , link there. according wikipedia , should possible (i couldn't find steinberg documentation relevant linux) can't find makefile or instructions on how build if aren't using xcode or visual studio. i'm vst 3 sdk doesn't support linux. when try compile plugin under linux, error: ./base/source/fatomic.cpp:39:30: fatal error: libkern/osatomic.h: no such file or directory this issue caused following code in "vst3 sdk/base/source/fatomic.cpp" #if mac #include <libkern/osatomic.h> #if mac_os_x_version_min_required > mac_os_x_version_10_4 #define native_atomic_type (volatile int32_t*) #else #define native_atomic_type (int32_t*) #endif #elif windows #include <windows.h> #endif but hope compiling under linux work vst sdk 2.4. reading. jvstwrapper seems run on lin...
Read more

java - Output of Eclipse is rubbish -

i want write program in eclipse language different english. when display output using system.out program, displays output correctly in console editor. when accept input console , output it, output rubbish. how can solve it. many thnaks i'm not sure if problem, have character encoding problem. make sure you're using unicode, utf-8. you can read more @ official docs converting non-unicode text .
Read more

jquery - Confused with JSON data and normal data in Django ajax request -

i read json internet still have not got grasp of it. reading article http://webcloud.se/log/ajax-in-django-with-jquery/ i not understood first part function using json def xhr_test(request, format): if request.is_ajax(): if format == 'xml': mimetype = 'application/xml' if format == 'json': mimetype = 'application/javascript' data = serializers.serialize(format, examplemodel.objects.all()) return httpresponse(data,mimetype) # if want prevent non xhr calls else: return httpresponse(status=400) my main problems are from function getting format variable does format json mean data given function json or data recived json can give me simple example ouput of function data = serializers.serialize(format, examplemodel.objects.all()) how use data when response in jquery function if don't use json in above function how input , response chnage thanks ...
Read more