100 Commits
Author SHA1 Message Date
Alex Morcos 5b8b387752 Fix overly eager BIP30 bypass 2018-02-15 13:31:45 -05:00
Alex Morcos bf64c3cb34 Ignore transactions added to mempool during a reorg for fee estimation purposes. 2017-09-12 12:40:06 -04:00
Alex Morcos 04f78ab5b9 Do not reject based on mempool min fee when bypass_limits is set.
This should have always been the case, but we will correctly trim to size after
a reorg which is when bypass_limits is set.
2017-09-12 12:35:35 -04:00
Alex Morcos fd849e1b03 Change AcceptToMemoryPool function signature
Combine fLimitFree and fOverrideMempoolLimit into a single boolean:
bypass_limits.  This is used to indicate that mempool limiting based on feerate
should be bypassed.  It is used when readding transactions from a reorg and then
the mempool is trimmed to size after all transactions are added and they can be
evaluated in the context of their descendants. No changes to behavior.
2017-09-12 12:30:26 -04:00
Alex Morcos 6af49dddea Output a bit more information for fee calculation report. 2017-08-25 14:59:36 -05:00
Alex Morcos a54c7b94f8 Fix rounding errors in calculation of minimum change size 2017-08-25 14:59:35 -05:00
Alex Morcos bdd5543ad6 Clarify help message for -discardfee 2017-07-28 09:54:59 -04:00
Alex Morcos 49d903e696 Eliminate fee overpaying edge case when subtracting fee from recipients 2017-07-27 22:04:17 -04:00
Alex Morcos 2264236da0 Rename -usewallet to -rpcwallet 2017-07-19 15:44:20 -04:00
Alex Morcos f4d00e63f7 Add a discard_rate
Any change output which would be dust at the discard_rate you are
willing to discard completely and add to fee (as well as continuing to
pay the fee that would have been needed for creating the change).
2017-07-17 13:40:30 -04:00
Alex Morcos 06bcdb8da6 Convert named argument from nblocks to conf_target
in estimatesmartfee and estimaterawfee.  Also reuse existing bounds checking.
2017-07-17 10:52:14 -04:00
Alex Morcos 439c4e8ad5 Improve api to estimatesmartfee
Change parameter for conservative estimates to be an estimate_mode string.
Change to never return a -1 for failure but to instead omit the feerate and
return an error string.  Throw JSONRPC error on invalid nblocks parameter.
2017-07-17 10:52:14 -04:00
Alex Morcos b1385852ef Remove factor of 3 from definition of dust.
This redefines dust to be the value of an output such that it would
cost that value in fees to (create and) spend the output at the dust
relay rate.  The previous definition was that it would cost 1/3 of the
value.  The default dust relay rate is correspondingly increased to
3000 sat/kB so the actual default dust output value of 546 satoshis
for a non-segwit output remains unchanged.  This commit is a refactor
only unless a dustrelayfee is passed on the commandline in which case
that number now needs to be increased by a factor of 3 to get the same
behavior.  -dustrelayfee is a hidden command line option.

Note: It's not exactly a refactor due to edge case changes in rounding
as evidenced by the required change to the unit test.
2017-07-17 07:10:03 -04:00
Alex Morcos 11590d39b9 Properly bound check conf_target in wallet RPC calls 2017-07-14 23:41:40 -04:00
Alex Morcos fd29d3df29 Remove checking of mempool min fee from estimateSmartFee.
This check has been moved to the wallet logic GetMinimumFee. The rpc call to
estimatesmartfee will now no longer return a result maxed with the mempool min
fee, but automated fee calculations from the wallet will produce the same result
as before and coincontrol and sendcoins dialogs in the GUI will correctly
display the right prospective fee.

changes to policy/fees.cpp include a big whitespace indentation change.
2017-07-14 23:41:40 -04:00
Alex Morcos 2fffaa9738 Make QT fee displays use GetMinimumFee instead of estimateSmartFee
Remove helper function (CalculateEstimateType) for determining whether
estimates should be conservative or not, now that this is only called
once from GetMinimumFee and incorporate the logic directly there.
2017-07-14 23:41:37 -04:00
Alex Morcos 1983ca6cb3 Use CoinControl to pass custom fee setting from QT.
This fixes buggy behavior where we were temporarily setting and unsetting the
global payTxFee when trying to send a transaction with a custom fee from the
GUI. The previous behavior was inconsistent depending on the order of using the
RPC call settxfee and clicking various radio buttons in the sendcoinsdialog.
The new behavior is that transactions sent with the GUI will always use either
the smartfee slider value or the custom fee set on the GUI and they will not
affect the global defaults which are only for RPC and initial GUI values.
2017-07-14 23:40:33 -04:00
Alex Morcos 03ee701161 Refactor to use CoinControl in GetMinimumFee and FeeBumper
Improve parameter precedence in coin_control
2017-07-14 23:10:58 -04:00
Alex Morcos ecd81dfa3c Make CoinControl a required argument to CreateTransaction 2017-07-14 23:07:18 -04:00
Alex Morcos dd97a529ad Properly forbid -salvagewallet and -zapwallettxes for multi wallet. 2017-07-13 13:48:28 -04:00
Alex Morcos 0f402b9263 Fix rare edge case of paying too many fees when transaction has no change.
Due to the iterative process of selecting new coins in each loop a new fee is
calculated that needs to be met each time.  In the typical case if the most
recent iteration of the loop produced a much smaller transaction and we have now
gathered inputs with too many fees, we can just reduce the change.  However in
the case where there is no change output, it is possible to end up with a
transaction which drastically overpays fees.  This commit addresses that case,
by creating a change output if the overpayment is large enough to support it,
this is accomplished by rerunning the transaction creation loop without
selecting new coins.

Thanks to instagibbs for working on this as well
2017-07-11 12:20:41 -04:00
Alex Morcos 253cd7ec4f Only reserve key for scriptChange once in CreateTransaction
This does not affect behavior but allows us to have access to an output to
scriptChange even if we currently do not have change in the transaction.
2017-07-11 12:17:56 -04:00
Alex Morcos 5e3b7b5686 Improve error reporting for estimaterawfee 2017-07-10 20:07:17 -04:00
Alex Morcos 1fafd704da Add function to report highest estimate target tracked per horizon 2017-07-10 20:07:17 -04:00
Alex Morcos 9c85b91dc1 Change API to estimaterawfee
Report results for all 3 possible time horizons instead of specifying time horizon as an argument.
2017-07-10 20:07:13 -04:00
Alex Morcos f135923ee2 Add RPC options for RBF, confirmation target, and conservative fee estimation.
Add support for setting each of these attributes on a per RPC call basis to sendtoaddress, sendmany, fundrawtransaction (already had RBF), and bumpfee (already had RBF and conf target).
2017-07-10 12:40:02 -04:00
Alex Morcos f0bf33da83 Change default fee estimation mode.
Fee estimates will default to be non-conservative if the transaction in question is opt-in-RBF.
2017-07-06 22:20:23 -04:00
Alex Morcos e0738e3d31 remove default argument from estimateSmartFee 2017-07-06 22:20:23 -04:00
Alex Morcos d507c301bc Introduce a fee estimate mode.
GetMinimumFee now passes the conservative argument into estimateSmartFee.
Call CalculateEstimateType(mode) before calling GetMinimumFee or estimateSmartFee to determine the value of this argument.
CCoinControl can now be used to control this mode.
2017-07-06 22:20:23 -04:00
Alex Morcos cfaef69ace remove default argument from GetMinimumFee 2017-07-06 22:20:23 -04:00
Alex Morcos 21d4afa12f Comment clarifications in coins.cpp 2017-06-27 16:08:06 -04:00
Alex Morcos 3c8a9aefff Add belt-and-suspenders in DisconnectBlock
These extra variables were previously checked before the move to per-txout database.
2017-06-27 15:04:56 -04:00
Alex Morcos 18bacec6c2 Make check to distinguish between orphan txs and old txs more efficient.
Checking for the existence in the CCoinsViewCache of the outputs of a new tx
will result in a disk hit for every output since they will not be found.  On the
other hand if those outputs exist already, then the inputs must also have been
missing, so we can move this check inside the input existence check so in the
common case of a new tx it doesn't need to run.

The purpose of the check is to avoid spamming the orphanMap with slightly old
txs which we have already seen in a block, but it is already only optimistic
(depending on the outputs not being spent), so make it even more efficient by
only checking the cache and not the entire pcoinsTip.
2017-06-27 14:14:15 -04:00
Alex Morcos 22378adbe5 Remove no longer used mempool.exists(outpoint) 2017-06-27 14:10:03 -04:00
Alex Morcos 61718268b5 Don't create change at the dust limit, even if it means paying more than expected 2017-06-14 15:34:19 -04:00
Alex Morcos e9cd7786da Pass in smart fee slider value to coin control dialog
Since cfe77ef41 the global nTxConfirmTarget wasn't being updated by the smart
fee slider and thus the coin control dialog and labels were not being updated.
2017-06-14 10:57:28 -04:00
Alex Morcos 1bebfc8d3a Output Fee Estimation Calculations in CreateTransaction 2017-06-13 12:35:04 -04:00
Alex Morcos 8e69adc665 Add missing include for atomic in db.h 2017-06-12 14:39:48 -04:00
Alex Morcos 525769853e Change semantics of HaveCoinInCache to match HaveCoin
Previously it was possible for HaveCoinInCache to return true for a spent
coin. It is more clear to keep the semantics the same. HaveCoinInCache is
used for two reasons:
- tracking coins we may want to uncache, in which case it is unlikely there
would be spent coins we could uncache (not dirty)
- optimistically checking whether we have already included a tx in the
blockchain, in which case a spent coin is not a reliable indicator that we have.
2017-06-08 16:56:50 -04:00
Alex Morcos 307013469f Populate services in GetLocalAddress
Previously if we didn't have any local addresses, GetLocalAddress would return
0.0.0.0 and then we'd swap in a peer's notion of our address in AdvertiseLocal,
but then nServices would never get set.
2017-05-18 16:57:53 -04:00
Alex Morcos 8d4f401269 Fix timestamp in fee estimate debug message 2017-05-18 12:38:43 -04:00
Alex Morcos 38bc1ec4a4 Make more json-like output from estimaterawfee 2017-05-17 15:42:32 -04:00
Alex Morcos 65d484adf9 Output line to debug.log when IsInitialBlockDownload latches to false 2017-05-11 13:12:42 -04:00
Alex Morcos 2d2e17052c Comments and improved documentation 2017-05-10 11:48:46 -04:00
Alex Morcos ef589f8d40 minor cleanup: remove unnecessary variable 2017-05-10 11:47:45 -04:00
Alex Morcos 3ee76d6de5 Introduce a scale factor
For the per confirmation number tracking of data, introduce a scale factor so that in the longer horizones confirmations are bucketed together at a resolution of the scale.  (instead of 1008 individual data points for each fee bucket, have 42 data points each covering 24 different confirmation values.. (1-24), (25-48), etc.. )
2017-05-10 11:47:45 -04:00
Alex Morcos 5f1f0c6490 Historical block span
Store in fee estimate file the block span for which we were tracking estimates, so we know what targets we can successfully evaluate with the data in the file. When restarting use either this historical block span to set valid range of targets until our current span of tracking estimates is just as long.
2017-05-10 11:47:45 -04:00
Alex Morcos aa19b8ea44 Clean up fee estimate debug printing 2017-05-10 11:47:45 -04:00
Alex Morcos 10f7cbd247 Track first recorded height
Track the first time we seen txs in a block that we have been tracking in our mempool. Used to evaluate validity of fee estimates for different targets.
2017-05-10 11:47:44 -04:00
Alex Morcos 3810e976d6 Rewrite estimateSmartFee
Change the logic of estimateSmartFee to check a 60% threshold at half the target, a 85% threshold at the target and a 95% threshold at double the target. Always check the shortest time horizon possible and ensure that estimates are monotonically decreasing.  Add a conservative mode, which makes sure that the 95% threshold is also met at longer time horizons as well.
2017-05-10 11:47:44 -04:00
Alex Morcos c7447ec303 Track failures in fee estimation.
Start tracking transactions which fail to confirm within the target and are then evicted or otherwise leave mempool.

Fix slight error in unit test.
2017-05-10 11:45:27 -04:00
Alex Morcos 4186d3fdfd Expose estimaterawfee
Track information the ranges of fee rates that were used to calculate the fee estimates (the last range of fee rates in which the data points met the threshold and the first to fail) and provide an RPC call to return this information.
2017-05-10 11:45:26 -04:00
Alex Morcos 2681153af3 minor refactor: explicitly track start of new bucket range and don't update curNearBucket on final loop. 2017-05-10 11:45:26 -04:00
Alex Morcos 1ba43cc0ec Make EstimateMedianVal smarter about small failures.
Instead of stopping if it encounters a "sufficient" number of transactions which don't meet the threshold for being confirmed within the target, it keeps looking to add more transactions to see if there is a temporary blip in the data.  This allows a smaller number of required data points.
2017-05-10 11:45:26 -04:00
Alex Morcos d3e30bca1b Refactor to update moving average on fly 2017-05-10 11:45:26 -04:00
Alex Morcos e5007bae35 Change parameters for fee estimation and estimates on all 3 time horizons.
Make feerate buckets smaller (5% instead of 10%) and make the 3 different horizons have half lifes of 3 hours, 1 day and 1 week respectively.
2017-05-10 11:45:26 -04:00
Alex Morcos 3edbd79a65 cleanup: reduce to one GetMinimumFee call signature 2017-04-26 15:54:23 -04:00
Alex Morcos c0a273f4c8 Change file format for fee estimates.
Move buckets and bucketMap to be stored as part of overall serialization of estimator.
Add some placeholder data so file format is only changed once.
Maintain 3 different TxConfirmStats with potential for different decays and scales.
2017-04-20 15:46:15 -04:00
Alex Morcos 68af651498 MOVEONLY: move TxConfirmStats to cpp 2017-04-10 13:56:50 -04:00
Alex Morcos 2332f19bef Initialize TxConfirmStats in constructor
and change to storing as a pointer.
2017-04-10 13:56:50 -04:00
Alex Morcos 5ba81e54e0 Read and Write fee estimate file directly from CBlockPolicyEstimator 2017-04-10 13:56:50 -04:00
Alex Morcos 14e10aa842 Call estimate(Smart)Fee directly from CBlockPolicyEstimator 2017-04-10 13:56:50 -04:00
Alex Morcos dbb9e3699b Give CBlockPolicyEstimator it's own lock 2017-04-10 13:51:51 -04:00
Alex Morcos f6187d6e39 Make processBlockTx private. 2017-04-10 13:51:51 -04:00
Alex Morcos ae7327b832 Make feeEstimator its own global instance of CBlockPolicyEstimator 2017-04-10 13:51:51 -04:00
Alex Morcos 1b55e07b7a Make threshold for flushing more conservative.
Always leave a reasonable buffer of 50MB for usage from newly connected block (once over 50%) and increase the high water mark buffer to 200MB.
2017-03-31 14:30:31 -04:00
Alex Morcos f33afd3b2b Lower default memory footprint slightly 2017-03-31 14:07:56 -04:00
Alex Morcos 5b95a190e8 Make pcoinsTip memory calculations consistent
Since we are more accurately measuring pcoinsTip peak usage at twice the current in dynamic usage, it makes sense to double the default (this will lead to the same effective usage and peak usage as previously).
We should also double the buffer used to avoid flushing if above 90% but still sufficient space remaining.
2017-03-31 13:15:39 -04:00
Alex Morcos b421e6ddcf Update example bitcoin.conf 2017-03-03 16:50:20 -05:00
Alex Morcos 7d4e9509ad Allow setting minrelaytxfee to 0
Setting minrelaytxfee to 0 will allow all transactions regardless of fee to enter your mempool until it reaches its size limit.  However now that mempool limiting is governed by a separate incrementalrelay fee, it is an unnecessary restriction to prevent a minrelaytxfee of 0.
2017-03-03 16:50:20 -05:00
Alex Morcos 359e8a03d1 [cleanup] Remove coin age priority completely.
Remove GetPriority and ComputePriority.  Remove internal machinery for tracking priority in CTxMemPoolEntry.
2017-03-03 16:50:20 -05:00
Alex Morcos f9b9371c60 [rpc] Remove priorityDelta from prioritisetransaction
This a breaking API change to the prioritisetransaction RPC call which previously required exactly three arguments and now requires exactly two (hash and feeDelta).  The function prioritiseTransaction is also updated.
2017-03-03 16:50:19 -05:00
Alex Morcos 49be7e1bef [rpc] Remove priority information from mempool RPC calls
"startingpriority" and "currentpriority" are no longer returned in the JSON information about a mempool entry.  This affects getmempoolancestors, getmempooldescendants, getmempooolentry, and getrawmempool.
2017-03-03 16:50:19 -05:00
Alex Morcos 0315888d0d [test] Remove priority from tests
Remove all coin age priority functionality from unit tests and RPC tests.
2017-03-03 16:50:19 -05:00
Alex Morcos f838005444 No longer allow "free" transactions
Remove -limitfreerelay and always enforce minRelayTxFee in the mempool (except from disconnected blocks)

Remove -relaypriority, the option was only used for the ability to allow free transactions to be relayed regardless of their priority.  Both notions no longer apply.
2017-03-03 16:50:19 -05:00
Alex Morcos ad727f4eaf [rpc] sendrawtransaction no longer bypasses minRelayTxFee
The prioritisetransaction API can always be used if a transaction needs to be submitted that bypasses minRelayTxFee.
2017-02-27 11:23:51 -05:00
Alex Morcos fe282acd76 [cleanup] Remove estimatePriority and estimateSmartPriority
Unused everywhere now except one test.
2017-02-27 11:23:51 -05:00
Alex Morcos 400b15147c [debug] Change -printpriority option
-printpriority output is now changed to only show the fee rate and hash of transactions included in a block by the mining code.
2017-02-27 11:23:50 -05:00
Alex Morcos 272b25a6a9 [mining] Remove -blockprioritysize.
Remove ability of mining code to fill part of a block with transactions sorted by coin age.
2017-02-27 11:23:50 -05:00
Alex Morcos 12839cdd56 [rpc] Remove estimatepriority and estimatesmartpriority.
The RPC calls were already deprecated.
2017-02-27 11:23:50 -05:00
Alex Morcos bc8fd12289 Remove harmless read of unusued priority estimates 2017-02-21 11:12:44 -05:00
Alex Morcos 91fb506e0a Add two hour buffer to manual pruning 2017-02-16 17:47:56 -05:00
Alex Morcos 4b189c1340 Change bumpfee result value from 'oldfee' to 'origfee'.
The result value indicates the actual fee on the transaction that was replaced. But there is an error message which uses the description 'oldfee' to refer to the original fee rate applied to the new transaction's estimated max size.  It was confusing that two different uses of 'oldfee' had two different numeric values.
2017-01-26 12:03:47 -05:00
Alex Morcos 0c0c63f70a Introduce WALLET_INCREMENTAL_RELAY_FEE
Have wallet's default bump value be higher than the default incrementalRelayFee to future proof against changes to incremental relay fee.  Only applies when not setting the fee rate directly.
2017-01-26 12:03:47 -05:00
Alex Morcos e8021ec919 Use CWallet::GetMinimumFee in bumpfee
Use the wallet's fee calculation logic to properly clamp fee against minimums and maximums when calculating the fee for a bumpfee transaction.  Unless totalFee is explictly given, in which case, manually check against min, but do nothing to adjust given fee.

In all cases do a final check against maxTxFee (after adding any incremental amount).
2017-01-26 09:43:00 -05:00
Alex Morcos ae9719ab87 Refactor GetMinimumFee to give option of providing targetFee 2017-01-25 22:24:40 -05:00
Alex Morcos 29fb311858 Increase minimum debug.log size to 10MB after shrink. 2017-01-24 20:28:00 -05:00
Alex Morcos 094e4b3383 Better document usage of SyncTransaction 2017-01-23 15:43:22 -05:00
Alex Morcos 4afbde6028 Introduce MemPoolConflictRemovalTracker
Analogue to ConnectTrace that tracks transactions that have been removed from the mempool due to conflicts and then passes them through SyncTransaction at the end of its scope.
2017-01-23 15:43:22 -05:00
Alex Morcos de1ae324bf Exclude RBF txs from fee estimation 2017-01-20 15:04:16 -05:00
Alex Morcos fe8e8efcf9 [rpc] Add incremental relay fee to getnetworkinfo 2017-01-20 07:40:36 -05:00
Alex Morcos bd92f2481c [bugfix] save feeDelta instead of priorityDelta in DumpMempool 2017-01-19 23:45:02 -05:00
Alex Morcos 6b331e6cf9 Fix to have miner test aware of new separate block min tx fee 2017-01-19 21:35:39 -05:00
Alex Morcos de6400de5d Fix missing use of dustRelayFee 2017-01-19 20:31:29 -05:00
Alex Morcos 5b158707f2 Use incrementalRelayFee for BIP 125 replacement 2017-01-19 20:31:29 -05:00
Alex Morcos ad82cb06ce Remove unnecessary min fee argument in CTxMemPool constructor 2017-01-17 13:21:35 -05:00
Alex Morcos 2a7b56cc0e CBlockPolicyEstimator now uses hard coded minimum bucket feerate 2017-01-17 13:21:31 -05:00
Alex Morcos ac9d3d25f7 Change fee estimation bucket limit variable names 2017-01-17 13:19:32 -05:00
Alex Morcos eb30d1a5b2 Introduce -dustrelayfee 2017-01-16 08:40:40 -05:00
Alex Morcos 7b1add3c28 Introduce -incrementalrelayfee 2017-01-16 08:39:03 -05:00