Modify

Ticket #2832 (closed Bugs: wontfix)

Opened 5 years ago

Last modified 3 weeks ago

Asio sync IO functions need timeout parameters

Reported by: Stephan Menzel <stephan.menzel@…> Owned by: chris_kohlhoff
Milestone: Boost 1.39.0 Component: asio
Version: Boost 1.38.0 Severity: Showstopper
Keywords: Cc: stephan.menzel@…, ronny.spiegel@…

Description

Hi Chris,

knowing this is an issue that has been discussed again and again I think it's better to handle it in a ticket.

Long story:  http://article.gmane.org/gmane.comp.lib.boost.user/45360

Short story: I think asio's sync routines either need a timeout parameter or need to be removed.

Sync IO without a timeout is something I can hardly imagine any sensible use case for. In most cases one would definately need such functionality. Workarounds have been suggested but most would create too much performance or boilerplate overhead within the application. This should be handled within the lib and would actually render the sync routines a lot more useful.

Thanks a bunch!

Stephan

Attachments

Change History

comment:1 Changed 4 years ago by anonymous

Hi Chris,

any chance, this may get some attention soon?

Cheers,

Stephan

comment:2 Changed 4 years ago by chris_kohlhoff

  • Status changed from new to closed
  • Resolution set to wontfix

Just because you can't imagine a use case doesn't mean there aren't any :)

I don't agree with either of the suggestions:

1) Remove the synchronous functions. They may be useful to applications that are signal aware, that use non-blocking i/o, or just plain don't care. They're in POSIX this way so they're staying in asio this way.

2) Add timeout parameters. It's the wrong place in the API to add a timeout as it does not scale to higher levels of abstraction i.e. composed operations. Timers are kept separate so that they may be equally used with composed operations such as async_read or async_read_until as with the socket's async_read_some operation.

I'm interested in exploring and improving these use cases, but in my opinion these are not viable solutions.

comment:3 Changed 4 years ago by stephan.menzel@…

  • Status changed from closed to reopened
  • Resolution wontfix deleted

Hi Chris,

I guess I have to work through this disappointment then.... And yet, if there are no use cases and the situation is good as it is, how come the topic pops up in both asio and boost mailing list with such a regularity?

My code has now several unportable spots in one lib caused entirely by this. Whenever things have to go synchronous, I have to workaround that way:

Example snippet UDP request/answer:

asio::ip::udp::endpoint sender_endpoint;

asio::ip::udp::socket socket(m_iosrv, asio::ip::udp::v4()); socket.send_to(asio::buffer(requeststring), m_pwh_master_ep);

asio::ip::udp::socket::native_type native_sock = socket.native(); fd_set rfds; FD_ZERO(&rfds); FD_SET(native_sock, &rfds);

struct timeval tv; tv.tv_sec = 42; tv.tv_usec = 0; if (!select(native_sock + 1, &rfds, NULL, NULL, &tv)) {

THROW(net_error, "timeout reading from socket");

}

size_t reply_length =

socket.receive_from(asio::buffer(reply, udp_max_length), sender_endpoint);

...and so forth and suchlike. Is that really what you want asio code to look like?

Realizing this with a timer would be OK for me and I tried but this doesn't work because the timers cannot cancel syncronous operations. Behold your very own docs: http://www.boost.org/doc/libs/1_41_0/doc/html/boost_asio/reference/basic_datagram_socket/cancel.html

It clearly says "cancel all asynchronous operations" so I can't timeout a sync_read with it.

Once I go in these synchronous functions asio appears not to offer any kind of defined way out of it. Please correct me if I'm wrong but all the answers to all the questions on the lists always boiled down to this.

If the given solutions are not viable, well that's alright with me. Just tried to help. But in any case the topic needs to be addressed.

By the way, If you say "They're in POSIX this way so they're staying in asio this way.", can you clearify that? In my example I have to go down to select, which has a timeout and solves the problem. Besides, I doubt very much this is a valid point. The whole boost is about offering stuff that should be cross platform standard but is not yet. Does POSIX offer regexes, time localisation, string formatting or parsing? Boost offers lots of such things and if plain POSIX is enough and everything on top of it is unnecessary, what's the point of having asio or indeed boost anyway?

Sorry for being so stubborn and verbose but I urge you to reconsider this. I'm sure there must be a way.

Cheers, Stephan

BTW: Milestone 1.39, so take your time, no rush ;-)

comment:4 Changed 4 years ago by Ronny Spiegel <ronny.spiegel@…>

  • Cc ronny.spiegel@… added

comment:5 Changed 4 years ago by anonymous

  • Status changed from reopened to closed
  • Resolution set to wontfix

I suspect you misinterpreted my response. To clarify:

  • I did not suggest there are no use cases for a timed read in a synchronous context.
  • I did say there are use cases for synchronous reads and writes, and so I will not consider removing them (as the trac item suggests).
  • My reference to POSIX is that it also provides *non-timed* synchronous operations, such as read() and write(). They are the equivalent of asio's synchronous functions. They are used in similar circumstances.

If you need cancellability, then my answer will always be that you should use asynchronous operations. I am interested in improving techniques to make it easier to use asynchronous operations in synchronous call contexts. But as this trac item is predicated on changing the behaviour of the synchronous functions, I'm closing it.

P.S. your select()-based code is broken :)

comment:6 Changed 4 years ago by anonymous

Well, obviously there's no point in discussing this any further.

Nevertheless it would have been a good way of discussing this to not simply poopoo all of my arguments despite the obvious truth in them but simply state a way to solve this very real and existing dilemma. Thus giving proof of it's nonexistence. Again, if the situation is good as it is, why am I having such problems with it. If you consider blocking synch operations without a timeout such a good thing to have there's nothing I can add.

Greetings...

Stephan

comment:7 Changed 4 years ago by stephan.menzel@…

Ah, and by the way, nothing could prove my point any better than your post scriptum so thanks for this statement.

comment:8 Changed 3 years ago by stephanecharette@…

We're recently begun investigating re-writing the networking parts of our app using ASIO, but this is a blocker for us. We can get most of the select+read calls replaced very easily with ASIO synchronous calls, but we're stuck trying to figure out how to get the select timeouts implemented. Using asynchronous calls is more work than it would be for us to port our own networking code to the necessary platforms we need. After discussing in StackOverflow? and getting pointed to this bug ticket, we decided to back out of the ASIO port we began last week. Synchronous network calls are great and simple to use especially since they map relatively well to legacy socket code, but the missing timeouts from legacy select() calls make ASIO unusable for porting existing apps with ease.

At the very least, careful explanation of this in the docs or a better explanation (+ examples) of whatever we missed would probably be greatly appreciated. Otherwise, with the exception of this one tiny feature, we were impressed by how "clean" our C++ socket code felt once we'd converted parts of it to ASIO.

comment:9 Changed 2 years ago by anonymous

Tried the async timer approach, but could never get that to work on our application which has to read from one thread and reply on the same socket on another thread. Yeah, tried strands, but it wouldn't work.

So synchronous would be nice, simply because synchronous UDP operations are supposed to be thread safe.

comment:10 Changed 23 months ago by klein

@stephan.menzel Awesome code! The only one i found in the freaking internet, was looking for that timeout code like madness, thanks! Helped really a lot! And yeah it would be much better if there was a timeout option directly in boost syncronous udp socks, then it would be much more better than using the native approach.

comment:11 Changed 21 months ago by anonymous

Why don't you just try to add or explain why you can't do that, instead of poopooing the arguments.

Many people need such a feature and it's not about if having synch. functions with no timeouts have a use. You can basically make this an optional parameter.

comment:12 Changed 19 months ago by fredrik

I'd like to second klein's comment and than Stephan for the code snippet. I need to be able to have timeouts in my legacy synchronous code without the trouble of rewriting the whole thing to async I/O.

It's too bad that the asio maintainers are so disinterested.

comment:13 Changed 18 months ago by anonymous

I'd like to also throw in my 2 cents in favor of having this added in for synchronous sockets. There is at least one use case where it's far easier to simple have a synchronous socket with a timeout than do the async method. For example, I'm writing a sort of local DNS server right now where I need to forward out requests to an external nameserver in certain cases. So essentially I have a simple setup where I have a single service running and a async handle send/onSent method.

So in each of these threads is where I must decode and read the incoming request and choose to forward it or not. In order to use the async way I need to basically have a socket running on a different service so when I call run() on that service, it's blocking until the async receive with timeout completes. If I set that socket up (for external resolution) to use my existing io service, guess what, I can use the run() call to be blocking within the current thread... so easy solution is to have a synchronous call for external resolution... but then since I can't put a timeout on it, if one call fails my whole program locks. Bad. We could argue for days about how to change the structure of my program to be "better".... or we could simply have a synchronous timeout and then everything works beautifully and logically. Now I've got to come up with platform specific hacks to make this work....

Don't get me wrong, open source is great, the work is much appreciated. As someone who has open sourced tens of thousands of lines of code in my career, I understand the frustration when someone whines about my hard work that I've given away for free.. but emotions aside, if they're right, they're right.

comment:14 Changed 18 months ago by ascensionsystems@…

Sigh, in my previous comment as "anonymous" I made a mistake, should be:

"guess what, I CAN'T use the run() call to be blocking within the current thread..."

Thanks.

comment:15 Changed 15 months ago by anonymous

Timeout is needed

comment:16 Changed 15 months ago by Stephan Menzel

Well, things have been quiet around here and I would like to add a few words since so many of you have commented here.

I have reconsidered my original opinion. I believe now it was part wishful thinking and part misconception of what asio is. I have been using boost for a long time now and I remember, in the first of those years I often thought: All this is great but what's seriously missing here is networking. No matter how much boost principles, types and libs were weaved into every part of my software I was always forced to drop out and become platform dependent or DIY when it came to networking. And I happened to do most of my work with networks. So when I stumbled across the first versions of asio (not part of boost then) I was excited and adopted early on. Later, it would become part of boost and I thought: Finally! Boost has gotten its networking! And this is how I thought about boost.asio, leading me to above statements. And yet, this was wrong. It has always been and still is "ASIO" as in "asynchronous IO" and not boost.network or whatever. So Chris suggesting to be asynchronous whenever this topic came up was right from his perspective. When you wanna do async io, you may choose this but when you wanna do generic networking sync and async, asio probably ain't what you are looking for. Sadly, this would imply that boost still lacks generic networking but I wouldn't go this far. There have been efforts with cpp-netlib but I don't really see that happening. As a result of this ticket here I have reduced the extent of asio's usage in my projects. Away from "using asio and its types whenever it comes to networking" to "using ASIO behind the hood when it comes to asychronous operations". In this regard, ASIO is still an awesome lib. Nowadays, I mostly use the io_service class directly for thread multiplexing and de-multiplexing. It never let me down and is well documented. A little networking now and then but not as much as I used to. I now often go native for that. I still believe that in the brutish realities of commercial software development it is not option to simply expect to go asynchronous everywhere. Large scale legacy products need refurbishing and maintenance, blocking third party calls will force constraints on your architecture as well as decisions made in the past that may be gradually changed towards an asynch architecture but rarely in one big haul. So exitsing synchronous parts and components will still need to have their place and pure async is certainly not the answer to everything.

So after all I'm OK with asio not implementing those timeouts. I still believe the sync operations are useless without it but I respect the "a" in "asio" and don't have to use it. Let's see if cpp-netlib or anything like that make it into boost eventually. Would still be good to have sync and async networking in one go. As for asio, it became such an important part of all I do with threads and actions and such... Wouldn't miss it.

Stephan

comment:17 Changed 15 months ago by y

  • Severity changed from Problem to Showstopper

well, I admit I'm a complete boost::noob.

I found myself in this situation: had to modify some ASIO code writted by someone else. But timeout is absolutely needed.

I have to tell you, sadly the solution that required less study/dev time was: drop asio, use my own socket class.

please, reconsider. you could use some #ifdef directive if you don't want dependences asio<->timers or whatever else, but in 2k13 a socket class without timeout is not exactly "useful"

comment:18 follow-up: ↓ 19 Changed 14 months ago by anonymous

This is probably the dumbest ticket discussion I ever read.

Everybody who is using blocking socket functions wants a timeout for them. But no, there is a better solution, the "official" one: simulate a blocking function by using asynchronous functions and waiting for them to finish.

Wow. Talk about scratching your left ear with your right foot. I don't know about you guys, but Computer Science 101 taught me that when encountering a problem a program should stop and handle it, not wait and wait and wait some more for it to solve itself.

Connect or read functions which take minutes to return are just plain stupid. They should timeout and do it fast, not wait or retry forever.

PS: If you like the asynchronous approach so much, why don't you create some overloads for the synchronous functions which do what this tries to explain.

PPS: Synchronous and asynchronous are very very different. Stop suggesting asynchronous solutions for synchronous problems or designs.

comment:19 in reply to: ↑ 18 Changed 14 months ago by stephan.menzel@…

Replying to anonymous:

This is probably the dumbest ticket discussion I ever read.

Anonymous, if you sit it out way past computer science 101 eventually they also tell you that a professional and polite tone in a discussion generally yields a higher chance of success.

With this being said, a discussion with no involvement of the only person who can actually do something about it, Chris for example, is indeed kinda pointless.

comment:20 Changed 13 months ago by anonymous

Maybe instead of adding timeouts to sync calls a new synchronous function "select" could be added that would take a list of SyncReadStreams? and SyncWriteStreams? and a timeout value.

comment:21 Changed 12 months ago by Magallo

Timeout is OBVIOUSLY needed!

comment:22 Changed 12 months ago by Magallo

Could some possibly post a sample code, using select() on native sokcet? Someone similar to the code posted by stephan.menzel@… some comments above (see comment 4) but working?

comment:23 Changed 10 months ago by SlowRiot

There are definitely places where either simulating a sync timeout using async functions, or resorting to platform-specific socket timeouts, is absolutely necessary. This is true of both new and legacy code. Unfortunately, being the sole network library in Boost, asio has too great a level of exposure to get away with ignoring this under the excuse that it's a specifically async library; asio provides those sync functions, but intentionally hobbled versions, and this is irresponsible.

Since it's possible to currently implement a native socket timeout with only a few lines of extra platform-specific code on top of asio, and many people are using this already, i can't really see any valid justification for not including that functionality in asio itself, when it would clearly serve the purpose that all boost is intended for - avoiding platform-specifics in common and advanced programming tasks, just like this one.

comment:24 Changed 10 months ago by stephan.menzel@…

Very well spoken, @SlowRiot?. With this bug now entering it's 4th year, without Chris responding in any way to the discussion, I have to wonder if boost asio is actually maintained. Maybe it would be better to do the ol' DIY and patch the thing. A patch could be posted on the mailing list and if it's rejected there at least it's a little more out in the open than this ticket. The question might get some attention. After all, there have been very few comments here agreeing with the absence of said timeouts. Is there anyone reading this who could assist in that? Maybe one could team up and do it? If there's reaction to this I will set up a temp email address and we can get in touch.

comment:25 Changed 4 weeks ago by anonymous

I, too, would like to see timeouts on sync calls - I made the mistake of writing using sync. Now it's causing a problem and I have to change it all to async. I don't see many practical "real world uses" for sync without timeouts.

I just started working with Boost, this missing feature is by far the most disappointing part of the library.

comment:26 Changed 3 weeks ago by helge@…

Although Asio has "asynchronous" in its name it is extremely useful for synchronous use, too. Writing solid networking code is not trivial and Asio simplifies the process a great deal - reducing both time and probability of error.

For that reason I second the request for timeout parameters for the synchronous functions.

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.