Hodl Invoices (Breaking Bitcoin training session)

https://twitter.com/kanzure/status/1146124260324384768

My name is Joost, Michael introduced me already. I’ve been working with Lightning Labs since the middle of last year and during that time I’ve worked on several subprojects within lnd. lnd is the Lightning implementation developed by Lightning Labs. I’m not sure really what the background knowledge is at this point in this three day course. One of those subprojects within lnd is called hodl invoices and this is the topic that I would like to talk about today. I’ve got a few slides prepared but I’m also going to draw on the whiteboard and take questions, make sure that it all lands within the model that you all currently have of how Lightning works and perhaps also how lnd works. If you’ve got any questions feel free to interrupt and ask, we can discuss things. Also if I trigger something that is a little bit related to what we’re talking about I’m happy to go into that as well when we’re talking about these topics. So hodl invoices. I can explain this on different levels. Obviously there is the why. Why do we have hodl invoices, what can you do with it? There is the hodl invoice on the level of the protocol so what’s happening on the network layer if you hodl invoice. There is also something to be said about the way we actually implemented this in lnd and the different trade-offs that are there, work that is still to be done there because we’ve basically just released hodl invoices in a minimum shape. There is a lot to improve on that.

https://github.com/lightningnetwork/lnd/pull/2022

I will go over to the whiteboard and draw a basic diagram. So a hodl invoice, basically it is nothing new. It is actually the same as a regular invoice. The only thing that is different is the timing. The timing when the invoice is actually settled. To understand how this works suppose we have a three hop route. This is the sender of the payment, it sends a HTLC to hop 1, hop 1 forwards the HTLC to hop 2 and hop 2 forwards the HTLC to the final node. If I say HTLC does everyone know what it is at this point? So this has been introduced. So what normally happens is that the sender sends the HTLC, hop 1 knows that given the preimage they can pull this money over so they are willing to hand out their HTLC and in this way they will chain until they get to the final destination. What normally happens is that when it gets to the final destination the HTLC chain has arrived at the point where the preimage is actually known. So the idea is that all these guys don’t know what the preimage of the HTLC is because if they would they would actually have the payment. If you make a payment tied to a hash to which the intermediate node know the preimage, your HTLC might never reach the intended destination. For example, this node could say “Alright I’ve got the preimage, I’ll pull in the money. I’m not going to forward anymore” and basically you’ve lost your money to someone else. It is important that the preimage is secret. Normally in Lightning the final destination of the payment only knows the preimage. Once the HTLC gets there, this final node is going to disclose the preimage, pull in the money and by doing this it atomically reveals the preimage to this hop 2. Using the preimage it also pulls the money from hop 1 and hop 1 pulls the money from the sender. That makes the circle round and the payment has been completed. This is something that happens completely automatically so inside the node software this HTLC is received, there is a database that contains all the invoices and the preimages that belong to those invoices. What the recipient does is it does a lookup, it finds the preimage and it pulls in the money. You don’t need to do anything with that. You don’t need to be at home, you need to be online of course but you don’t need to be at home, it is all automatic. The only thing that has really changed here with a hodl invoice is that the receiver R is not immediately pulling in the money. He might know the preimage but he is not yet pulling the money to allow a certain window of his size that can be used to decide whether you actually want to have this money. A greedy recipient wouldn’t need hodl invoices because it would always pull the money when it gets there. But there are situations where you would want this. For example, you are a business, you are accepting a payment but you don’t want to accept any payment because if you accept any payment it could be that people call your customer service and they ask for a refund. It is all extra work, you don’t really want that. Another reason why a business wouldn’t want to get a payment sometimes is if a customer places an order and the goods are not in stock. He made the payment already, we go to check the inventory and at that point we’re sold out. It would be not so convenient to make a refund especially a refund over Lightning because with Lightning you may not know the actual person who sent the money. This is obfuscated for the recipient. A refund is something that you want to avoid in that case. This is where the hodl invoice comes in because what you can do with the hodl invoice is hodl on, that’s the joke, hodl on to the HTLC and decide do I actually want this payment. Yes or No? You check your inventory. If you’ve got your goods you reveal the preimage, you pull in the money and in the end the sender will have paid. And if not you would just cancel the payment. If you cancel the payment these intermediate nodes would not get their forwarding fees. As you probably now know, these hops want to have a part of the payment as compensation for their service. If you cancel a payment you don’t reveal the preimage and the payment isn’t happening but the fees also aren’t happening. So basically cancelling a hodl invoice is free for the sender and free for the recipient. Only those intermediate nodes, you could say that they have some kind of a cost because they have locked their funds to a payment that has been hodled by the receiver for some time. After it has been cancelled they get nothing.

Q - The intermediate nodes I presume can’t tell the difference between a hodl invoice and a regular invoice?

A - It is not even in the onion blob. Nobody can really see this except for the recipient. There is an invoice, this encoded payment request. At the moment there is not a field or flag or anything to communicate this might be held, this HTLC. On the protocol level there is also nothing to communicate this. For these intermediate nodes it looks like any other payment. Once they’ve forwarded it they start to figure out it takes a bit long so maybe it is a hodl invoice but it could also be other reasons why it is slow.

Q - A hodl invoice encumbers people’s Bitcoins temporarily in HTLCs. So how do we prevent a DOS attack?

A - I will get to that point later. There are ways to detect this but it is important to know that hodl invoices are not something new. Hodling a HTLC has always been possible. The only thing we do here is expose this as useful functionality. Something that can happen in Lightning, we call these black holes. You extend your HTLC and then you hear nothing back. Maybe not for an hour, maybe not for a day, maybe not for two weeks. At some point it will pass the HTLC expiry height which will force a channel close. All this time you don’t really know what is happening. This existed already, hodl invoices have always existed. You could argue that offering these kind of things on the command line, possibly also in UIs, makes it easier for people to play around with this and behave in a way that not all routing nodes might like. I think the most important thing is to recognize that this possibility has always existed, it is there in the protocol. So if you want to do something about it, preventing a hodl invoice from being created is not the fix. It should be something on the protocol level, maybe senders need to pay for nodes hodling, there are different ideas there but not without their trade-offs. With Lightning you need to be really careful that nothing can be gamed. If you make people pay for something and routing nodes figure out that they can on purpose hodl for example, make someone pay more. It is not so easy to do that.

Q - It says “may or may not know the preimage”. I understand the case where it doesn’t know the preimage. But in the case when it does surely that node can just go onchain with a valid transaction as soon as it discovers the preimage?

A - It could try to discover it on its own but normally it discovers it because its own HTLC is settled offchain.

Q - Generally, the pre-image is shared without it going onchain. So it’s only in the case when the channel is closed where the preimage would be announced onchain?

A - This has nothing to do with hodl invoice but that is how it works. If a channel closes the party who has the HTLC and wants to settle it, it is creating a Bitcoin onchain transaction that contains the preimage and then the nodes extract the preimage from the onchain transaction and use it to settle back towards the sender. Probably just offchain because if one channel closes all the others can stay open. There are two ways to learn the preimage. It can be a Lightning protocol message that contains the preimage or in the non-happy flow and the channel closing, this is a preimage that is discovered in the onchain tx. The way you started your question about the final destination, no we will not necessarily know the preimage. These are indeed two variations of it. The example that I just gave on the inventory check, in that case the recipient knows the preimage. It created the preimage itself, it communicated the hash of the preimage through the invoice to the buyer of the goods. And then it uses as a hook inside the process to decide whether he actually wants to have this money or not. I’ve got on the next slide some examples of different use cases where they don’t know the preimage. It enables some interesting things. I’m wondering maybe I should do the demo first. I think I’ll make a little demo. What I wanted to do was to create a hodl invoice on my Lightning testnet node and then pay it with my app but my app is not functioning yet so I think I will do it CLI unless someone has a testnet app on their phone? Not yet. It will become more technical but it is no problem I guess.

It might be difficult to see what is on the screen. I’ve got my Terminal windows there. What I’m going to do is start on regtest three instances of lnd. It is Alice, Bob and Charlie. They are not connected Alice -> Bob -> Charlie but it is Bob -> Alice -> Charlie in this case. What I want to do is have Charlie create an invoice and make Bob pay the invoice. Bob will pay through Alice to Charlie. I will first do it with a normal invoice and then I will show you what it looks like with a hodl invoice. On the CLI for Charlie I’m going to add an invoice, let’s say 6000 satoshis. What it gives me is a payment request. A payment request is all the invoice details encoded and signed by the person who created the invoice. What I can do is lncli-alice decodepayreq and you can use this command, I don’t know if you’ve looked at this already, to decode payment requests. You can see what is in this payment request. There’s the destination there, there’s the payment hash. The payment hash indicates the hodl invoice, this is the thing where the recipient has the preimage for and may or may not decide to use it. The amount, there are several other fields like cltv_expiry. I don’t know if you talked about this. For invoices in Lightning, when they get to the recipient they have a requirement of a minimum blocks still left to the expiry of the HTLC. What you don’t want to do is accept a payment, send out the pizza and then find out you only have one block left before it expires so you really need to hurry to go to chain and hope you will claim it. If you need to hurry to go to chain, the other party might also try to play the timeout transaction and you get involved in a race. In general there’s a broadcast window, that’s what we call this, we don’t want to accept any HTLCs that are already inside the broadcast window. We want to have a convenient period of time in which we can claim the HTLC onchain if needed and during this window our counterparty can’t claim yet. If they want to play the timeout they need to wait until the expiry. With this value you can control how many blocks you still want to have left until expiry. The default value in lnd is 40. This means that when an invoice is accepted and settled, we know there is still 40 blocks left. If something goes wrong with our channel we’ve got enough time to claim the HTLC. With regular invoices it is usually left to the default but with hodl invoices this can be quite useful because you can put a much higher number in here. The window that you have to do your additional actions can be chosen by the recipient. If you have an inventory check for which you need to go into the warehouse and walk around to see if what you need is there, you might say “40 blocks is not enough, I need a full day.” Using this value you can configure how much time you have after your sender offers the HTLC to decide what you want to do with it. Sometimes a short period is enough but it could also be really long. This is something that some people are worried about because they say “People are going to create hodl invoices. They’ll make a 1000 block timeout and maybe they just forget about it or they make it a custom to sell them only at the very last minute. All this time our capital is locked up in a chain of channels.” For the demo it is set to 40. This is the payment request. If I go to Bob and I pay the invoice it gives me a confirmation. I click yes. What it shows me, this is not hodl invoice specific, just explaining for context. It will show me the preimage because I made a payment and in exchange for the payment they gave me the preimage. This is what I really bought. It shows me the route that was taken to complete the payment. This was a regular invoice so it settled automatically. If you’re at home or not it doesn’t really matter, it will settle. Now I’m going to make a hodl invoice. It is the same command for Charlie. I do addholdinvoice. I need to pass in the hash that it should be locked against. I just explained that sometimes the recipient doesn’t know the preimage himself so if I wouldn’t pass in anything here what could lnd do? It could only generate a preimage but then it would know the preimage. The way this is designed as a caller of this function is that you always have to provide a hash yourself to tie the HTLC to and that way you can also be sure that lnd itself is never able to settle without you supplying the preimage because we are not even passing the preimage onto the command line. I’ve got a little tool here, it is called genhash and what this does, you can run it several times, is generating preimages and hashes. I use this for testing. I’m going to generate a combination of hashes and preimages. So this hash is the hash of this number. And now I’m going to create a hodl invoice type to this hash. The final parameter is an amount. It generates a new payment request and if I decode this one it will more or less show me the same things. There is nothing in here to indicate that it is a hodl invoice, it is just a regular payment request. As a sender you don’t really know whether your payment is going to be hodled. This is something a site that sells something could actually mention like “Be aware. If you pay to us we hodl your payment for a while so don’t panic if you don’t get the preimage immediately.” Now I can go to Bob and I can pay the invoice. I confirm. What you see is that nothing is happening really because in the protocol there is only a single round trip. Those HTLCs are extended and at some point there is a failure or there is a settle. The settle or the fail is bounced back to the sender. There is no other way to do some intermediate communication. You can’t see for example an ACK that the recipient actually got the payment. On the command line I can look at that. If I run on Charlie listinvoices you will see that this 7000 invoice, I guess this is the same payment request. So we can see that Charlie actually accepted this payment. We have the HTLC, Bob can do nothing about this anymore, there is no way to cancel this for him. The HTLC is with Charlie and Charlie can decide what to do with this. How to decide this is something that happens on the command line. Before I do that I want to explain the difficulty here is that Bob, he has nothing back. In the happy flow this is what is expected. It could also be that the payment never reached Charlie and this is the difficulty. Bob doesn’t really know what is going on now. Did it get there? Did it not get there? Because we don’t have an in protocol ACK to communicate this, users of this functionality would use some out-of-band communication channels. Possibly on a website they show “We are hodling your invoice, it is with us, it is alright and we will settle soon.”

Q - The happy flow is when the payment has successfully routed and the preimage is shared along that route right back to the sender?

A - Yes that’s the full happy flow but now we are halfway through the happy flow which means that the payment arrived at the recipient but we haven’t decided what to do with it yet. Therefore this window is still blocking here.

So then for Charlie there are actually two commands on the CLI. It is called settleinvoice and for settleinvoice I need to pass in the preimage. So lnd doesn’t know the preimage so you need to pass in the preimage otherwise it will never be able to pull the HTLC. If I do this you will see that Bob unblocks and he gets the payment confirmation. It looks exactly the same as a regular invoice, it is just the timing is different. There is this hook with a delay. If I do the same thing with another hodl invoice but with a different hash. I will pay this, -f means no confirmation so it is hodled again. I can do listinvoices for Charlie and the other thing that I can do is I can cancel the invoice, lncli cancelinvoice. I need to find my hash because with cancelling you don’t need to deal with the preimage, maybe you don’t even know the preimage yourself. Cancelling is done using the hash. If you cancel this Bob will get a non-payment hash error. There’s not really an error in which we can say “we got it but we cancelled anyway” so we use one of the existing errors for this. It is in our payment hash so it looks like the recipient didn’t know the invoice but actually it did know it but it cancelled it afterwards. So how is this explanation so far connecting with what you all know?

Q - What does the -f flag do? I haven’t seen that before.

A - It is force. Otherwise it says these are the details of the payment, confirm, yes or no? You have to type yes and enter. With -f it is just force. It is not important really.

This is how it works with hodl invoice. Actually it is pretty simple, nothing changes at a protocol level as explained. It is just the timing is different. I think what makes this interesting is the applications you can do with this. If I go back to the slides. I think I explained most of the things here. The maximum hold, I spoke about this briefly. In the invoice you set a delta value, how many blocks do you want to still keep as a recipient when you accept the invoice payment. When blocks keep coming in and get close to the expiry you need to cancel. If you wouldn’t cancel then your channel peer will say “You’re holding this HTLC but you’re not settling, you’re not cancelling so I’m going to close the channel with you.” The maximum hold is defined somewhere close to the expiry of the HTLC that arrived. You can control this yourself by changing that final CLTV delta in the invoice payment request.

So the refunds, I already spoke about this. Another thing is anti-DOS, a fidelity bond sort of. If you have a web service and you’re worried about people DOSing your service or behaving in a bad way otherwise you could ask them to pay to you a hodl invoice for a really small amount. You give them a payment request, they make the payment, they get some kind of API key and they start using the service. But if you detect abuse using this API key what you can do is pull their payment and then cancel their API key. So they will be punished with a chosen amount, it should be low enough so that users are willing to pay it, to trust you with this money. For anti-DOS this can be really low because with DOS you need to generate lots of traffic. If they don’t behave well, your users, you will pull their money. After some time let’s say you get close to the expiry value, maybe 1 day or 1 session, it depends on how you define this. You can just cancel the payment and then the user of your service didn’t pay anything.

Q - How theoretically long could this hodl invoice remain? Indefinitely?

A - It depends on the node software really. Nodes can decide for themselves, what’s the maximum HTLC that we want to forward. They can accept any HTLC they want because it is not their risk. Once they start to forward they get an instruction in the onion blob, how many blocks their forwarded HTLC should be locked for. They have their own idea about what they think is reasonable there. In lnd at the moment we have a hard coded limit of 5000 blocks which is pretty long really. This is something to think about if people are using payments with 5000 blocks all the time, you maybe won’t like it anymore after some time. It could be that this is somethings that can be communicated in the channel policy. Currently channels have a policy that defines the minimum value that they can relay, the maximum value, the fee to pay. You could also say “We have a channel, there is a maximum timelock for this channel but maybe we have another channel that allows longer locks but the fee is higher.” The problem is that in the end if the payment is cancelled you will earn nothing in both cases. For successful hodl invoices you could adjust your fees to some expected timelock. For cancelled ones, nobody will pay anything in the end so you provided your service for free there and maybe for 5000 blocks.

With that one application, it is quite interesting because it is a real problem. With this you can add some money into it, it should only be a small amount but it can be very effective in deterring people from trying to use your service in a way that is not intended. The normal user is paying a small amount but it will be refunded at the end, close to the expiry. It is quite interesting. The other thing is atomic swaps. Atomic means that you pay and you get something in exchange for that and this is an atomic operation. Those two steps can’t be separated. In the real world it is not always completely like that but you will see what I mean when I start explaining this. One atomic swap in the real world could be a customer ordering a pizza. What he could do is he could order a pizza on a website and he would get a Lightning payment request from the pizza restaurant. The pizza restaurant actually created this invoice as a hodl invoice and they don’t know the preimage themselves. So the customer pays this invoice and the pizza restaurant sees it has got the payment for the pizza now. If I had the preimage I would be able to get this money. The delivery person goes to deliver it, rings the doorbell and he says “This is your pizza. Can I have your preimage otherwise I won’t hand over the pizza to you.” This is pretty nice because the customer generated this preimage, passed the hash to the pizza restaurant and only in exchange for the pizza will it get the preimage. So the pizza restaurant knows that this person has the money because we are having it. The person ordering the pizza knows for sure that he will only pay if he actually gets the pizza, maybe with additional requirements like it should be hot for example. This thing can happen offline. Suppose you order the pizza for tomorrow. You already paid the hodl invoice, you take the preimage and you go up somewhere high in the mountains, an isolated place. The pizza comes, it is maybe a little artificial, you just need to exchange the pizza for the preimage. The person delivering the pizza can actually calculate the hash of the preimage to verify offline. You don’t need the internet for that, whether this is actually the preimage that’s connected to the pizza. You can have your pizza there and if they don’t deliver because they think it is too high up the mountain you pay nothing. I thought this was quite interesting.

Q - The key differentiator there is that you’re able to make a payment for the pizza even though you are offline. That’s the key difference because I could receive the pizza and pay the Lightning payment as I receive it.

A - Yes you could do that but then the pizza delivery man is not sure that you actually have the money. Suppose you order a pizza and then he comes up to you and you say “Sorry I have no money.” With the hodl invoice you’ve already paid so it is not a guarantee that the pizza person will actually be paid but you’ve committed your funds to that pizza already.

Q - It is like a two step payment. It is like you’re putting your funds up in an escrow? The recipient can’t take your money?

A - They can’t because they don’t know the preimage. As a customer you generate your own preimage, you only tell the hash to the pizza restaurant. They know those funds are there.

Q - The commit doesn’t force you to do anything. When you pay you commit to the fact that you own the money but it doesn’t constrain you to do anything?

A - Once you’ve paid the hodl invoice that money is not available anymore for other things, it is locked during that time. But of course if the pizza comes to you and you say “I don’t want it anymore” and I don’t give you the preimage it is still a conflict. The number of different types of conflicts that can happen is lower because the thing that I have no money at all, that’s not possible anymore. You can do the same thing with three parties, almost the same. There is a sender and he sends a parcel to a recipient. The delivery service creates a hodl invoice for the cost of delivery and the hodl invoice is paid by the sender but the delivery service doesn’t know the preimage so it can’t pull. They need to go to the recipient of the parcel, the recipient of the parcel doesn’t need to pay anything but they do have the preimage. The delivery service wants to get paid for the delivery so they go to the recipient and they ask for the preimage because if they don’t get the preimage they don’t get paid. When they get the preimage they can prove to the sender that they actually delivered the parcel because they can show the preimage so the recipient must have given the preimage to them. It is almost the same as the pizza but there are three parties involved. The party paying is a different party to the one receiving the package. The third thing is almost like a chain of HTLCs in Lightning but in the real world. There are again three parties involved. There’s a customer who orders from the web shop, there’s a web shop who sends the product using the delivery service and the delivery service is delivering to the customer. So the customer pays the web shop with a preimage so the web shop doesn’t get the money. Then the web shop pays the delivery service with the same hash, tied to the same hash. The delivery service also doesn’t know what the preimage is so nobody has paid anything yet. Then the delivery service goes up to the customer, the customer sees the product that he ordered. He will give the preimage to the delivery service, the delivery service wants to get paid so they pull the money. They reveal the preimage to the web shop and the web shop is pulling the money for the product from the customer. It is not completely trustless because the delivery can be much cheaper than the product itself so people can steal things along the way. The whole idea of opening up the chain by revealing the preimage by the customer, I find this quite interesting. I’m not sure how soon this will be adapted but these are constructions that you can’t easily make with traditional payment systems. There is more trust involved, there’s more refunds going on. Here, the customer, they know that if they don’t reveal the preimage nobody can take their money. Nobody is holding it in custody at the moment, I only release it when I get the product that I really ordered. From a customer perspective it is pretty safe.

Q - To me it sounds like a pre-payment?

A - Yes it is a pre-payment but with a pre-payment will they give it back to you? In this case it is a pre-payment but they can’t just take it. You can decide yourself whether they actually can get this pre-payment.

Q - Could you give them the preimage and they don’t have to cash it in yet and they could cancel the preimage?

A - If you give it to them they can take the money whenever they want but they don’t have to. I showed in the demo you’ve got these two commands, lncli settle and lncli cancel and you can use them to either cancel or settle even if you know the preimage.

Q - It is kind of like the feature credit cards have, the chargeback. Instead of sending cash and not being able to retrieve it back, this gives you the option for you to retrieve the money back but it is split up differently to credit cards?

A - Yes I think with a credit card there is a 30 day money back guarantee while here you can decide not to pay but once you reveal the preimage the deal is done. That is maybe a bit more strict but easier for a merchant to know that I’ve delivered, now I’ve got my money and I don’t need to wait for another 29 days if something is not according to a customer’s liking and he wants a refund or chargeback.

In the online world there is also an application, I don’t know if you talked about this already, Lightning Labs offers a service called Lightning Loop and you can use it to change the balance of your channels. In Lightning if you open a channel you are the only one funding this channel and all the balance of this channel is on your side so you can send payments from that but you can’t receive. In order to receive you first need to send which is not very intuitive for people used to regular onchain or fiat or all kinds of payments. Normally there is not really a limit to what you can receive, it is not a problem to receive money. With Lightning that is different. You open your channel and if you don’t do anything else you can’t receive. With Lightning Loop there are two variations of this. There is Loop Out and Loop In. The Loop Out allows you to push a balance out. What you do is you’ve got a channel, all of the funds are on my side and I can decide to push half of what is in this channel to the other party. After I do this I am able to receive through this channel because there is a balance on the other side and they can forward payments to me using that balance. You could make a donation so you pay a random person some money and then you’ve got your balance but then you lost that money. What Loop does is it takes this money from you and it refunds it to you onchain. You send a Lightning payment and it loops back to you via an onchain transaction. In Loop Out we are also using hodl invoice to make this trustless. The way this works is you make that offchain payment to move the balance in your channel but the Loop server doesn’t know the preimage of this payment. It holds the payment but it doesn’t know the preimage. Then it is going to publish an onchain transaction, it is also a hash locked contract. Then the user of the service can sweep that money to its wallet but in order to sweep it needs to reveal the preimage. What the Loop server does is it watches the chain, it sees that its transaction is getting spent, it looks in the transaction and figures out what the preimage is. It uses it to pull the offchain payment. That makes the service trustless and it uses hodl invoices to do so. There’s another example which I heard of, I think it was demonstrated at the Lightning Hack Day in Munich. It is an auction. I’m not completely sure if I tell it right but I looked at the idea and how it should work. There’s an auction, people need to bid but if you bid the auctioneer wants to know that you’ve actually got the money. So he creates hodl invoices and people are paying those hodl invoices. When a higher bid comes, the lower bid is cancelled. How this would work is there is a QR code on the screen, people can scan this, it can send the payment. You see a blocking screen, nothing is happening. It probably means you are the highest bidder at that point. Someone else comes in, he overbids you and then you see your payment failed, you know that you need to pay more to win the auction. Also quite interesting. Then you’ve got in the gaming area, there is a person called MandelDuck and he created a game. It is a two person shooter really where you have to place bets. The person who shoots the other, maybe I’m inventing a part of it, it is not completely how it works. It could work like this. The person who shoots the other gets the bet. What you don’t really want ideally is a gaming server that is holding all the money from all those bets. What you can do with hodl invoice is that you have those two players send a hodl invoice to each other but both of them are unaware of the preimage. They commit a bet to each other and nothing is happening yet. Then they play the game through a game server and the person who wins will win the preimage that allows him to pull the money from the loser. The game server is not holding any funds, it is just knowing the preimages and revealing only one when the game is finished.

Q - You’re trusting the server to assess who’s the victor and then release the preimage to the victor?

A - Yes. As I understand it, those games they often use a game server anyway to enforce the rules of the game. There is no actual money being held there. Even the amount of the bet could be unknown the game server.

Q - It is a very similar scenario to the 2-of-3 escrow but it seems more flexible. A 2-of-3 multisig. Onchain if you’re trying to do something similar in the betting scenario or in the buying a pizza scenario, you’d pay money to a 2-of-3 multisig and one of those keys would be the oracle or the moderator.

A - That’s similar only this is instant. Onchain, we all know the problems with onchain. What fee to choose, waiting for 24 hours to confirm etc.

Q - It also seems more flexible for some reason. It’s not just a one-to-one mapping, it seems an improvement on the 2-of-3 scenario. There’s only two people? I suppose there are only two people. You are still relying on the server to provide the information to determine who gets the preimage. So they’re not directly involved in the funds in terms of having a key to the multisig but they are still providing information that determines who gets the preimage.

One final application, this is something that I have been experimenting with myself. I don’t know if you know the Lightning Torch? The idea was we’re passing along a payment that increase in value every time. I wanted to be part of this history. I created my invoice for the Lightning Torch but I actually made it a hodl invoice. What I wanted to do was accept the torch and then look for my successor. I wanted to get an invoice from that person, pay that invoice first so I know that I passed on the torch and only then pull the money. If for some reason I couldn’t find a next person I could cancel the payment, I could say “I had nothing to do with it. I accepted the money but I cancelled it.” It could even be such that in this case I knew the preimage myself but the next carrier of the torch only knows the hash. You could do a trustless hop. It is not like the Lightning Torch is anything that happens all the time but you could have a person who you don’t really trust. I think it happened that one or two people stole the torch but in the end they gave it back. So if there is someone you don’t really trust, maybe they only have five followers on Twitter, can you do a hodl invoice and tell me who the next person is? You give the preimage to the next. He can be part of the torch but there’s no way for him to steal the money because he can only get the preimage once he has paid himself and then pull the incoming payment. It is nothing really practical but a funny fact.

Q - Did you use that as part of the Lightning Torch? So you were one of the participants? There’s a hodl invoice as part of the Lightning Torch? Who was the person who you didn’t want to trust after you in this chain, on the Lightning Torch chain? They just had five followers?

A - I just wanted to promote my hodl invoice. I thought I could use the Lightning Torch as a vehicle to do so.

Some concerns, some of them we already spoke about. If you have a hodl invoice you need to watch that timeout because lnd in this case is not going to do it. If you create a hodl invoice, someone pays it and you don’t act, your channel peer will at some point close the channel with you and cancel for you. That’s something you don’t want to let happen. If you create an application that builds on hodl invoice you need to make sure that you keep up with blocks and know when your hodl invoice needs to be decided upon. Either cancelled or settled, both is fine, but you need to act, it is not done automatically.

Q - Do we close it onchain?

A - No not onchain. For that particular HTLC you need to decide cancel or settle offchain. Because otherwise it goes onchain and it will be cancelled.

Q - Do you cancel the whole channel onchain if the HTLC gets held up for too long?

A - That is what your peer will do. In general, this is nothing to do with hodl invoice.

Q - My next question was going to be for regular payments if the HTLC gets lost and you don’t receive an error message your node might close your whole channel onchain?

A - You are going to close yourself. If you extend a HTLC it is your funds on the line. So when you get close to the expiry it will soon happen. Once it happens you immediately publish a transaction to reclaim your funds. You do this onchain because if your peer is not responding anymore you can’t update the commitment transaction offchain. If you are the receiver of a HTLC or a receiver of a payment through hodl invoice your peer will exit with this behavior. They will watch the height and when your expiry height gets close, maybe ten blocks before or so, they will start closing channels. You need to be aware of that. It is not the end of the world if your channel closes, you’re not losing any money but it does cost a bit. You have to reopen it again, maybe this is a channel with a long history, a year old channel. Other implementations have started to work channel age into their pathfinding heuristics. They favor old channels as they have proven that they could remain open for a while so they are probably more reliable. There is something to say about not letting a channel get closed. Something we want to do ourselves is create an automatic process there so that when you get close to the expiry, ideally lnd should cancel the invoice for you because it will be cancelled anyway if you don’t act. You might as well cancel it yourself just in time to prevent channel closure. How the application is dealing with this is another thing because if the application still thinks it is being held, the money is still with you, you go out to bring the pizza and you get the preimage and you come back home and you can’t claim the money. The application should still be aware of this but this is like a safety precaution to prevent channel closes.

The other concern is feedback to the sender. I showed in the demo, you make a payment, nothing is happening so you don’t know what is going on. Is it ever going to happen or not? You can’t also send it again. At the moment you’re pretty much relying on having an external channel communicating to you that everything is ok. Whether something will be added to the protocol is still under discussion because what do you with the situation where you pay but the acknowledgement that it actually was accepted is not received by you. In some way you are always pushing forward the problem in the end. Someone can always say they never received it and there is no way to prove this. We are brainstorming a little bit about this but the workaround for now is to do it on a website or via some other protocol to tell the user that they’re good.

Q - Out of curiosity what about hashlocking the product in some way? For example I want to sell you software and we have this setup. You do the bond and I lock the software, encrypt the software in such a way that the preimage is used to decrypt. Once you open the software the bond gets paid because you reveal the preimage.

A - I’m not sure how that would work. There’s probably something in there encrypting with the preimage but I’m not sure if it would work in that case. As a buyer I get the software, I have the preimage myself, I can unlock the software. I can never tell the shop what the preimage was.

Q - I suppose you know you’ve received some software but you don’t know if it is the software that you sought to buy until you’ve decrypted it and been able to look at it.

One other concern is locking up of liquidity, we already talked about it a bit. There’s 20x leverage really because a route in Lightning can be 20 hops. If you create a hodl invoice yourself and then you pay yourself through a 20 hop route, all those nodes along the route are committing capital for the payment. You could try to use the maximum accepted expiry so 5000 blocks for example or maybe it is a little lower but still a lot. Then you do nothing. With your 1BTC you’ve locked 20BTC of the network that can’t move during that time. It is something that you can do at the moment. It is to be seen how this develops. Currently conditions are very friendly on the network, there is not much adverse behavior going on. It is a factor that does exist. It is difficult to defend against. You can lower your maximum acceptable timelock value but even then after it expires the attacker can send out the same thing again with a renewed HTLC. I’ve no answer to that, I don’t know how that is going to develop.

Q - I’m sure you’ve considered this but what about parallel HTLCs. So you want to do this hodl invoice, I know it is a hodl invoice, you tell me it could be 1-7 days for example. I say “Great. I’m going to charge you 10 satoshis per hour and I would like a parallel HTLC where every hour you pay me and as soon as you stop…”

A - That has been discussed as well, I need to have some kind of parallel channel to do this. You would think that in the end you would need such a thing if people can lock up someone’s money for free there needs to be some kind of monetary compensation for that.

One other problem is failure attribution. Failure attribution is if you make a payment it can be either be a successful payment or a failed payment. It may not be ideal. A failed payment is never ideal because you don’t want a failed payment. But a successful payment can also not be ideal if it was for example slow. It could be a non hodl payment that was slow in the end. There is one node that holds the payment for let’s say a day. In the end it still relays the payment to you. During this day you don’t know what’s going on, you made the payment and you hear nothing back. If you are at a bar maybe you ask for a new invoice to get the beer so you pay another invoice. After one day you finally get a confirmation and it says payment settled so you actually paid twice. This is something that you don’t want which is also not ideal. Then you’ve got failed payments that also have a delay. You hear nothing back and after a day it is a failure with a specific message. A failure has a source normally so every failure is signed by the sender of the failure but it doesn’t necessarily need to be the person who delayed. It could be that a node generated a failure and along the backward path somebody delayed this failure. The sender only sees this is when I sent it out, this is when I got it back. I don’t know really what happened. It is already a problem but if you also add into this mix hodl invoices it becomes even more unclear where the delays are actually happening. This is only a little bit related to hodl invoices but this is something else I’m working on at the moment. I’m investigating options to approve this, to get more feedback if you make a payment from where it gets stuck which node delayed it and do it in such a way that nodes can’t really avoid the judgement of the sender. Once you know the node that caused the problem you can apply a penalty and if you apply a penalty it won’t be as likely that you will use this node next time. But hodl invoices is a complication there because not every slow payment is a non-ideal payment. If the recipient caused the delay it can be ok.

This is the last slide. Atomic multipath payments. You have been speaking about this, multipath payments? You can already do now a non-atomic multipath payment, non-atomic in the sense that the recipient doesn’t need to wait for all the shards of the payment to come in. What you could do is you can make a hodl invoice and then you make multiple payments to this hodl invoice and they will all be hodled, nothing is paid. When the recipient sees that now I’ve got enough, they are all connected to the same hash, I can supply the preimage to the node and then everything gets pulled. It is a non-atomic multipath payment. Each of these shards can take different paths but the same difficulties apply such that the sender doesn’t really know whether its shards arrived or were stuck along the way. Also currently with lnd it is not possible to pay to the same hash twice. If you would want to simulate this you would need different nodes that can all pay to the same hash. The sender has its hodl invoice, it will accumulate those payments. It also doesn’t know what the toggle is so there are a lot of rough edges there. The basic idea of a non-atomic payment could be played with a little bit in that way. The only thing there is that the receiver doesn’t know the preimage of those until the very last shard. The idea there is that the last part of your payment contains an encrypted preimage that can be used to unlock, there’s some crypto going on there. That’s the basic idea. We are making payments where the receiver can only pull all of those payments when it receives the last one of those. This is what I prepared for hodl invoice.

Q - You must be familiar with purse.io and companies that let users sell stuff online and typically you deposit money with that company and that company acts a third party escrow. Do you think that would be the most obvious use case, getting rid of the whole third party escrow and just using hodl invoices for online payments?

A - I wouldn’t dare say how this is going to develop, I just see the possibility there. The workflow of a customer generating a hash, passing it to the web shop and then the web shop starting the delivery process without really having the money yet. At the moment they get paid first and then they deliver but as you say with Purse it is different, there is an escrow in between.

Q - Could I borrow something from a person with hodl invoice? I would like to borrow your car and you say “Ok but let’s put some money in escrow in the event that something happens with the car.” Could you see some way we could do that?

A - There needs to be an arbitrator there to decide whether there is a problem or not. If there are only two parties it won’t solve this problem because then you pay the bond but if you still have the preimage and you bring it back with a dent and you say “I didn’t do it” and I don’t give you the preimage. It isn’t exactly the same as when there is a third party. Possibly a third party can keep that preimage and decide whether to release it or not.

Q - Is this a good system for trial periods? You try something for ten days and then if you don’t cancel the payment comes through?

A - Yes it sounds like it could work for that.

Q - You were saying this could be used for cross-chain atomic swaps as well?

A - Yeah for Loop Out. With these submarine swaps we use it there to first get the offchain payment from the user and only once the user received the onchain payment they disclose the preimage and we use it to pull the offchain.

Q - What if it was hashlocked but not timelocked such that if the channel closes no one gets the money?

A - Nobody wants to hand out a hashlocked contract that is not timelocked because then if your peer disappears the money would be lost forever.

Q - Let’s say it was just me and you, there were no hops. You still need the timelock even in a payment channel.

A - You can set it to a really high value so it is almost the same as if there wasn’t a timelock. It is a risk because you need to trust the other party to reveal the hash at some point.

Q - If the counterparty disappears you’ve got no way of claiming your funds. What if we did this? I borrow your car, your car is worth $1000. You make me bond $2000 and there’s no timelock but what we do this is if I wreck the car you want $1000 and we’ve locked up $2000 and so we negotiate from there. It is like “Listen if you don’t want to pay me it is hashlocked forever. I lose $1000 from the car and you lose $2000. But if you unlock then we can negotiate where I give you $1000 back and I get a $1000.” An overcollateralized situation.

A - But the one who pays this risked $2000 at that point.

Q - One of them has got more at stake than the other so it is not an even negotiating position because one of the parties has more to lose than the other so you’ve got a stronger bargaining position. You said the ability for lnd to monitor the hodl invoice won’t be integrated anytime soon. Do you have any thoughts on how that could work where I don’t have to monitor the timelock outside of lnd. lnd does that for me.

A - How that could work, we just need to build it. Some code that is triggered every block that goes through all of the invoices and looks for the ones that are close to expiry for which we don’t have the preimage and cancels them. But it isn’t really a solution because in the end those HTLCs are held for a reason and the application that is holding them also needs to act. So basically if it happens to you it is a sign that something is not right in the application or in the design of the application. The application should keep track of those and act on them in a timely manner.

Q - When it is built, whether it is included in lnd or not will not just be whether people see that there is a use case for it, there is also a trade-off in terms of the attacks that it opens up by including it in lnd?

A - It is not really an attack because it is something that you can also code yourself really. In general there is so much work to do, many things need to be prioritized and so we thought with hodl invoice as it is now, people can do what we think they should be able to do. This automatic cancellation is sort of an add-on on top of it so it needs to compete with twenty other things that we want to do.

Q - It is just a case of priorities rather than concerns over whether it should be merged into lnd or not?

A - Yeah, definitely. We are a small team and there is too much work.

Q - Do you think that the narrative of why Lightning is useful is a good narrative because right now it is about micropayments but what you are talking about here is much more interesting than micropayments at least from my point of view. It is much more diverse. What is exciting about Lightning?

A - For me, hodl invoices I like because they are new use cases but for me a basic payment is already pretty exciting because it is a way to get around the onchain limitations. Fees and confirmation time, scalability in general. With Lightning, I’ve got one channel and we’ve got a meetup locally where I live and every time I go there I can use the same channel to pay over and over again. There are people there who pay onchain and they are complaining about fees and conf times. People from payment processing are there as well and they are accepting zero conf but not with RBF. There’s lots of stuff going on but with Lightning the experience becomes much cleaner and faster and cheaper.

Q - From my perspective it is not even about the micropayments, micropayments are the only way to do it, if you want to do micropayments you can’t do it onchain. All of these things are above micropayments. We talked about the benefits of Lightning earlier, it wasn’t just micropayments.

A - There is a place for onchain as well if you want to send $1 million. You would need huge channels and they’d probably charge big fees. With onchain there is a fixed tx fee, there’s an amount where it becomes more attractive to do it onchain.

Q - With Loop In Loop Out, why can’t you just open two channels to two different places with everything on your side and then send half of each channel?

A - Both channels can’t receive. You can send out through both but they can’t receive. You would send out through one but you can’t go into the other. I could send $10 to you and you give me $10 cash. Then I changed the balance of my channel and I can start receiving. Loop is a way to do this trustlessly with an onchain transaction to bring the balance back to zero.

Q - Where is a good explanation of that?

A - There’s a blog post on the Lightning Labs blog about Lightning Loop Out. Loop In is available currently on testnet, we’re working to release that for mainnet as well.

Q - So Loop Out is available but Loop In is just testnet?

A - Only testnet. Loop Out is for mainnet. If you’ve got a node and you need receive capacity, maybe you want to try out can I be a routing node, these are some things people want to do, you could create a channel to the destination of your liking and then Loop all of it out so you can start receiving through that channel. This is also different to other services that exist in which you buy a channel. You’ve got these services where you pay and they open channel to you. You would get that channel from them which means that people paying to you through this channel and you yourself paying through this channel will be subjected to the fees that they charge. I could sell you a channel for not that much money but then I could start charging 10% of everything I forward to you. This is something you might not want. You don’t want to do this to your customers for example, having only one route that is so expensive to them. Also once you receive this money at some point you want to send it out yourself again and you’re also going to pay those forwarding fees. If you buy a channel you also need to look at whom am I buying it from, what is their policy? Of course policies can change but services might also build a reputation at some point. These are the fees that we charge and we guarantee them for this long for example. Another aspect with choosing your channel peer is how well they are connected themselves because it doesn’t really make sense to buy a channel for someone who is not really connected at all. You’ve bought a channel and still people can’t pay to you through that channel because your peer is not well connected. With Loop it is different because you open a channel to any peer you like. You can look up these lists of destinations or 1ML has some kind of metric for node rank, I think it is called. You open to any one of those, you can look at their forwarding fees, you can try to figure out how they are connected. Then you open a channel and you push out all your balance and you can start receiving payments. It is different, in a way more decentralized because people can choose all their own peers. While if there are three services that sell channels then everyone is buying channels from those services, those will become central nodes.

Q - You did a presentation on routing at Breaking Bitcoin. I’ve got questions on routing. So lnd currently uses Dijkstra’s for routing and c-lightning uses Bellman-Ford?

A - I think they recently changed but I’m not sure.

Q - I’m assuming there was a discussion at some point where it was like “These are the routing algorithms that we could possibly use and we’re going to use this one rather than this one”?

A - I was not involved. When I joined it was already decided and implemented. Dijkstra’s is a pretty logical choice for this. With Bellman-Ford the advantage is that you can have negative weights. In Dijkstra every edge needs to have a zero or a positive weight because otherwise you might get into loops, it keeps on looping. It can happen in any case where you get into loops with negative weights. It could be a route that gets cheaper and cheaper by going round. Bellman-Ford is able to deal with those negative weights. The idea with those negative weights is that you could encourage people to route in a different direction through your channels. If you’ve got a channel and you want to get it more balanced you could attract people to use your route by offering negative fees. It is an idea but it is not really happening at the moment, you can’t even communicate in the protocol at the moment that you’re charging a negative fee. That would be a reason to use Bellman-Ford. Another thing to be aware of is that Dijkstra itself is optimal but you need to use it in the right way. What we’ve been doing in lnd at least is we’ve been adding more constraints to the route like a maximum fee and a maximum timelock that you can specify during pathfinding. If you specify those constraints Dijkstra is not optimal anymore because if you only choose the lowest cost all the time and then you run into a limit it is not going to backtrack at that point even though it really should to find the optimal solution. The space to explore really explodes with all those limits. Currently we just accept as a limitation of Dijkstra, it is also a trade-off with execution speed because we don’t want to make it too slow. We are aware that the way we currently use Dijkstra is yielding sub-optimal results.

Q - Is the long term goal to do a blended approach of Dijkstra’s and the reputation system?

A - It is already a blended approach because Dijkstra is optimizing a cost and this cost for us is not only the forwarding fee that you pay but it is a combination of the forwarding fee and the timelock. Everything else being equal we favor routes that lock up your money for a shorter period. Then in addition to that we factor in a probability. It was recently merged, there’s a reputation management system that keeps track of previous payments. It analyzes the outcomes and tries to estimate the probability of every channel succeeding if used in a payment attempt. If you multiply all those probabilities it will give you the routing probability. For a successful payment every step of the route needs to succeed so we multiply probabilities. This is factored into the cost and is more like an abstract cost as well and that is used in Dijkstra to optimize the path.

Q - That’s building a local view of the reputation of nodes that you’re connected to? It is only based on payments that you’ve tried or probes that you’ve personally done?

A - Yes. Everyone is on their own. They all build their own reputation database which is kind of inefficient you could say but decentralization is also important to us. It will be interesting to see how this is going to develop because we’re now also storing that data, maybe you could export it. It is only a small step to people saying “I’ve made 1 million payments. You can have my dataset so you can bootstrap your node.” It will be pretty interesting to see how this is going to work. Maybe also supplement the learned probabilities with some external data. If this is a good node I could add it to my whitelist somehow and work it into the probabilities. It propagates through to pathfinding and we will use that node more often if it is not much more expensive for example.

Q - Like a 1ML.com API?

A - It could be that they supply some kind of a list. There is always the incentivization argument there and do we want to connect to a server that provides all the information?

Q - There could be competition. It wouldn’t have to just be one.

A - It could be multiple, you could have multiple and you weigh it. It is so early days. If you think about these kind of things. Before this merge lnd would only keep reputation for a single payment. For the next payment it would start over as if nothing has happened. We’re extending this now. We’ve made it configurable. You can actually say “I want you to to remember for an hour” or “I want to remember for two weeks.” You can tune how aggressive it should prune the graph based on your observations. This is really just a start. What is also an interesting question is how important this will be in the end? Currently this is pretty new and everyone is trying to find out how to run a routing node. It may be at some point everything gets more professional. Nodes will stay up much more and routing is not that important anymore because everyone who offers that service and is profitable guarantees a minimum service level. Then it doesn’t matter anymore. The same could be said for the size of the graph. Currently you can open a channel and you say it is public, your channel peer will just accept it and they don’t care. This is all advertised so the graph becomes pretty big. I’ve got a public channel from my phone and nobody can route through it. My app is not even running and it is just polluting the graph. I also tried to convey this in my Breaking Bitcoin talk, by the time that senders become more demanding in terms of node performance, nodes will start to realize accepting public channels from someone you don’t know is actually a risk to your own forwarding revenue because if you accept a channel you are going to advertise it, people are going to try to use it to get their payment through. If it doesn’t work, in a worst case they penalize you for a month and if lots of people are doing this, this public channel is going to bring down your reputation. It is interesting to see how this is going to develop. The senders are increasing their expectations and sending this to controls in their reputation management system. On the other hand nodes seeking profitability and stepping up their game, increasing the level of service that they offer. It needs to go slowly because as a sender if you demand perfect execution every time you would blacklist the whole network and you would never get anything through. It is an interesting dynamic to see how these two things will go hand in hand. Now in lnd we added some parameters which you can use to tune the reputation management. Of course we don’t really know what people set this to. We have a default, it is one hour but maybe people start setting this to a week or so. It is also a little bit out of our own control, how this is going to be used. People will seek an optimal value to make payments so they want to be aggressive as possible but not if it would lead to huge fees being paid because there are only two reliable nodes that charge like 5% forwarding fees.

Q - It’s not just the reputation of your own node you’ve got to worry about, it is the reputation of the peers you choose to connect to?

A - If you’re advertising a channel then you will be held accountable for it in the end.

Q - What are the downsides to minimizing the number of hops out of curiosity? I feel like the more hops you have the more possible…

A - Every hop is a risk that something goes wrong. The fewer hops the better. But also every hop adds an additional timelock because they receive a HTLC, they hand out a HTLC and they require some delta in between for their own safety so they don’t get into the situation where the next peer withholds the payment and they are too late to pull theirs. Every hop that you add adds something to the timelock. In the happy flow it doesn’t really matter because it will be settled instantly. But if it is not the happy flow, there’s a node that disappears from the network, it will be locked for that number of blocks. So the fewer hops the better in general. Unless you want to really obfuscate what you’re doing possibly. There’s not visible but if all the hops on your path are owned by the same person he can trace where your payment is going.

Q - What are your views on balanced channels? Should you aim to keep it generally 50:50 unless you have specific information otherwise? I know Alex Bosworth has talked about keeping it at least 25:75 ratio.

A - I think this connects to what I said before. Senders dictate the rules really. If a sender says “I expect nodes to keep their channels 25:75 so I will only search for paths that utilize a maximum of let’s say 25% of the channel capacity.” If the balance between 25% and 75% I should always be able to send a payment of 25% of the capacity because even it is 25% or 75% there is still enough reserve to actually forward this. I could modify pathfinding. Currently if a channel is 1BTC and I try to route 1BTC through it the chances of this actually working is really low because if the balance is different than everything on one side it will already fail. If we change this to only pick channels that have capacity at least 4 times as much as the payment you want to send then we expect nodes to keep this 25:75 balance. If not, if they respond to us with a failure we just blacklist the node for a whole month again. We as the sender have the power to shape the network, shape the rules that nodes need to comply to.

Q - You don’t actually know what the inbound, outbound capacity ratio is but you’re setting that expectation that if I send this amount that you’re able to route. If you aren’t then I’m going to blacklist you. At any point in time I don’t actually know that inbound, outbound capacity ratio.

A - You don’t know but you just expect them to have a 25% reserve at all times. You could say “This is a waste of liquidity because there is 25% tied up in the channel that is never used” but maybe that is the cost of reliable payments in the end.

Q - They could just be unlucky. They could just get a lot of payments routed through them in one direction?

A - They might want to open multiple channels. You can always find solutions to be sure that you always deliver successful payments every time. It is just what’s the cost of that. The cost is lower if you just fail some payments now and then so you can use the channel all the way down to close to zero.