Thread Rating:
  • 1 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Question about converting exponential values in DT
#1
Solved: 11 Years, 4 Months, 3 Weeks ago
Hello,

I am trying to set a value that is represented by exponent 2^63. QTP always displays 9.22337203685478E+18. I want to display the actual number 9223372036854775808. Is there any way to do this? Will I have to create a function for this? If so, how can this be accomplished?

Thank you.
Reply
#2
Solved: 11 Years, 4 Months, 3 Weeks ago
Code:
a=Replace(FormatNumber(2^63,0,0,0),",","")
msgbox(a)
Reply
#3
Solved: 11 Years, 4 Months, 3 Weeks ago
Thanks inborntester. But the number returned is rounded up the last digit
(9223372036854780000). I need the exact number 9223372036854775808.
Reply
#4
Solved: 11 Years, 4 Months, 3 Weeks ago
think, its native format conversion of VB compiler not able to optimise. Better you can develope you own Power function if its need is high. Logically it can ge possible with few for looks and array objects.
Reply
#5
Solved: 11 Years, 4 Months, 3 Weeks ago
Values in VBScript are accurate to 15 digits only. This should set you in the right direction (taken from http://www.windows-api.com/microsoft/VBS...stion.aspx )

Code:
Wscript.Echo AddInts(148991, 76561197960265728)

Function AddInts(lngValue1, lngValue2)
Dim V1Hi, V1Lo, V2Hi, V2Lo
Dim Temp1, Temp2, Carry, A1, A2, A3

V1Hi = Fix(lngValue1 / (10^10))
V1Lo = lngValue1 - (V1Hi * (10^10))

V2Hi = Fix(lngValue2 / (10^10))
V2Lo = lngValue2 - (V2Hi * (10^10))

Temp1 = V1Lo + V2Lo
Carry = Fix(Temp1 / (10^10))
A3 = Temp1 - (Carry * (10^10))
Temp2 = V1Hi + V2Hi + Carry
A1 = Fix(Temp2 / (10^10))
A2 = Temp2 - (A1 * (10^10))

AddInts = CStr(A1) & CStr(A2) & CStr(A3)

End Function
Want to fast track your QTP/UFT Learning? Join our UFT Training Course
Reply
#6
Solved: 11 Years, 4 Months, 3 Weeks ago
Thank you very much Ankur this worked for me!!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Overflow error while converting a Long value to an Integer Ankesh 3 4,912 10-05-2011, 11:16 AM
Last Post: Ankesh
  how to validate combobox values with testdata values 1981.madhu@gmail.com 3 6,107 09-09-2009, 06:53 PM
Last Post: Saket
  Checking if values of Listbox/Combo box on a web page the same with values in DB? robocom 4 8,499 07-08-2009, 01:52 PM
Last Post: robocom

Forum Jump:


Users browsing this thread: 1 Guest(s)