Coverage for aiocoap/message.py: 85%

Shortcuts on this page

r m x   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

227 statements  

1# This file is part of the Python aiocoap library project. 

2# 

3# Copyright (c) 2012-2014 Maciej Wasilak <http://sixpinetrees.blogspot.com/>, 

4# 2013-2014 Christian Amsüss <c.amsuess@energyharvesting.at> 

5# 

6# aiocoap is free software, this file is published under the MIT license as 

7# described in the accompanying LICENSE file. 

8 

9import urllib.parse 

10import struct 

11import copy 

12import string 

13from collections import namedtuple 

14 

15from . import error, optiontypes 

16from .numbers.codes import Code, CHANGED 

17from .numbers.types import Type 

18from .numbers.constants import DEFAULT_BLOCK_SIZE_EXP 

19from .options import Options 

20from .util import hostportjoin, hostportsplit, Sentinel, quote_nonascii 

21from .util.uri import quote_factory, unreserved, sub_delims 

22from . import interfaces 

23 

24__all__ = ['Message', 'NoResponse'] 

25 

26# FIXME there should be a proper inteface for this that does all the urllib 

27# patching possibly required and works with pluggable transports. urls qualify 

28# if they can be parsed into the Proxy-Scheme / Uri-* structure. 

29coap_schemes = ['coap', 'coaps', 'coap+tcp', 'coaps+tcp', 'coap+ws', 'coaps+ws'] 

30 

31# Monkey patch urllib to make URL joining available in CoAP 

32# This is a workaround for <http://bugs.python.org/issue23759>. 

33urllib.parse.uses_relative.extend(coap_schemes) 

34urllib.parse.uses_netloc.extend(coap_schemes) 

35 

36class Message(object): 

37 """CoAP Message with some handling metadata 

38 

39 This object's attributes provide access to the fields in a CoAP message and 

40 can be directly manipulated. 

41 

42 * Some attributes are additional data that do not round-trip through 

43 serialization and deserialization. They are marked as "non-roundtrippable". 

44 * Some attributes that need to be filled for submission of the message can 

45 be left empty by most applications, and will be taken care of by the 

46 library. Those are marked as "managed". 

47 

48 The attributes are: 

49 

50 * :attr:`payload`: The payload (body) of the message as bytes. 

51 * :attr:`mtype`: Message type (CON, ACK etc, see :mod:`.numbers.types`). 

52 Managed unless set by the application. 

53 * :attr:`code`: The code (either request or response code), see 

54 :mod:`.numbers.codes`. 

55 * :attr:`opt`: A container for the options, see :class:`.options.Options`. 

56 

57 * :attr:`mid`: The message ID. Managed by the :class:`.Context`. 

58 * :attr:`token`: The message's token as bytes. Managed by the :class:`.Context`. 

59 * :attr:`remote`: The socket address of the other side, managed by the 

60 :class:`.protocol.Request` by resolving the ``.opt.uri_host`` or 

61 ``unresolved_remote``, or the :class:`.Responder` by echoing the incoming 

62 request's. Follows the :class:`.interfaces.EndpointAddress` interface. 

63 Non-roundtrippable. 

64 

65 While a message has not been transmitted, the property is managed by the 

66 :class:`.Message` itself using the :meth:`.set_request_uri()` or the 

67 constructor `uri` argument. 

68 

69 * :attr:`request`: The request to which an incoming response message 

70 belongs; only available at the client. Managed by the 

71 :class:`.interfaces.RequestProvider` (typically a :class:`.Context`). 

72 

73 These properties are still available but deprecated: 

74 

75 * requested_*: Managed by the :class:`.protocol.Request` a response results 

76 from, and filled with the request's URL data. Non-roundtrippable. 

77 

78 * unresolved_remote: ``host[:port]`` (strictly speaking; hostinfo as in a 

79 URI) formatted string. If this attribute is set, it overrides 

80 ``.RequestManageropt.uri_host`` (and ``-_port``) when it comes to filling the 

81 ``remote`` in an outgoing request. 

82 

83 Use this when you want to send a request with a host name that would not 

84 normally resolve to the destination address. (Typically, this is used for 

85 proxying.) 

86 

87 Options can be given as further keyword arguments at message construction 

88 time. This feature is experimental, as future message parameters could 

89 collide with options. 

90 

91 

92 The four messages involved in an exchange 

93 ----------------------------------------- 

94 

95 :: 

96 

97 Requester Responder 

98 

99 +-------------+ +-------------+ 

100 | request msg | ---- send request ---> | request msg | 

101 +-------------+ +-------------+ 

102 | 

103 processed into 

104 | 

105 v 

106 +-------------+ +-------------+ 

107 | response m. | <--- send response --- | response m. | 

108 +-------------+ +-------------+ 

109 

110 

111 The above shows the four message instances involved in communication 

112 between an aiocoap client and server process. Boxes represent instances of 

113 Message, and the messages on the same line represent a single CoAP as 

114 passed around on the network. Still, they differ in some aspects: 

115 

116 * The requested URI will look different between requester and responder 

117 if the requester uses a host name and does not send it in the message. 

118 * If the request was sent via multicast, the response's requested URI 

119 differs from the request URI because it has the responder's address 

120 filled in. That address is not known at the responder's side yet, as 

121 it is typically filled out by the network stack. 

122 * It is yet unclear whether the response's URI should contain an IP 

123 literal or a host name in the unicast case if the Uri-Host option was 

124 not sent. 

125 * Properties like Message ID and token will differ if a proxy was 

126 involved. 

127 * Some options or even the payload may differ if a proxy was involved. 

128 """ 

129 

130 def __init__(self, *, mtype=None, mid=None, code=None, payload=b'', token=b'', uri=None, **kwargs): 

131 self.version = 1 

132 if mtype is None: 

133 # leave it unspecified for convenience, sending functions will know what to do 

134 self.mtype = None 

135 else: 

136 self.mtype = Type(mtype) 

137 self.mid = mid 

138 if code is None: 

139 # as above with mtype 

140 self.code = None 

141 else: 

142 self.code = Code(code) 

143 self.token = token 

144 self.payload = payload 

145 self.opt = Options() 

146 

147 self.remote = None 

148 

149 # deprecation error, should go away roughly after 0.2 release 

150 if self.payload is None: 

151 raise TypeError("Payload must not be None. Use empty string instead.") 

152 

153 if uri: 

154 self.set_request_uri(uri) 

155 

156 for k, v in kwargs.items(): 

157 setattr(self.opt, k, v) 

158 

159 def __repr__(self): 

160 return "<aiocoap.Message at %#x: %s %s (%s, %s) remote %s%s%s>"%( 

161 id(self), 

162 self.mtype if self.mtype is not None else "no mtype,", 

163 self.code, 

164 "MID %s" % self.mid if self.mid is not None else "no MID", 

165 "token %s" % self.token.hex() if self.token else "empty token", 

166 self.remote, 

167 ", %s option(s)"%len(self.opt._options) if self.opt._options else "", 

168 ", %s byte(s) payload"%len(self.payload) if self.payload else "" 

169 ) 

170 

171 def copy(self, **kwargs): 

172 """Create a copy of the Message. kwargs are treated like the named 

173 arguments in the constructor, and update the copy.""" 

174 # This is part of moving messages in an "immutable" direction; not 

175 # necessarily hard immutable. Let's see where this goes. 

176 

177 new = type(self)( 

178 mtype=kwargs.pop('mtype', self.mtype), 

179 mid=kwargs.pop('mid', self.mid), 

180 code=kwargs.pop('code', self.code), 

181 payload=kwargs.pop('payload', self.payload), 

182 token=kwargs.pop('token', self.token), 

183 ) 

184 new.remote = kwargs.pop('remote', self.remote) 

185 new.opt = copy.deepcopy(self.opt) 

186 

187 if 'uri' in kwargs: 

188 new.set_request_uri(kwargs.pop('uri')) 

189 

190 for k, v in kwargs.items(): 

191 setattr(new.opt, k, v) 

192 

193 return new 

194 

195 @classmethod 

196 def decode(cls, rawdata, remote=None): 

197 """Create Message object from binary representation of message.""" 

198 try: 

199 (vttkl, code, mid) = struct.unpack('!BBH', rawdata[:4]) 

200 except struct.error: 

201 raise error.UnparsableMessage("Incoming message too short for CoAP") 

202 version = (vttkl & 0xC0) >> 6 

203 if version != 1: 

204 raise error.UnparsableMessage("Fatal Error: Protocol Version must be 1") 

205 mtype = (vttkl & 0x30) >> 4 

206 token_length = (vttkl & 0x0F) 

207 msg = Message(mtype=mtype, mid=mid, code=code) 

208 msg.token = rawdata[4:4 + token_length] 

209 msg.payload = msg.opt.decode(rawdata[4 + token_length:]) 

210 msg.remote = remote 

211 return msg 

212 

213 def encode(self): 

214 """Create binary representation of message from Message object.""" 

215 if self.code is None or self.mtype is None or self.mid is None: 

216 raise TypeError("Fatal Error: Code, Message Type and Message ID must not be None.") 

217 rawdata = bytes([(self.version << 6) + ((self.mtype & 0x03) << 4) + (len(self.token) & 0x0F)]) 

218 rawdata += struct.pack('!BH', self.code, self.mid) 

219 rawdata += self.token 

220 rawdata += self.opt.encode() 

221 if len(self.payload) > 0: 

222 rawdata += bytes([0xFF]) 

223 rawdata += self.payload 

224 return rawdata 

225 

226 def get_cache_key(self, ignore_options=()): 

227 """Generate a hashable and comparable object (currently a tuple) from 

228 the message's code and all option values that are part of the cache key 

229 and not in the optional list of ignore_options (which is the list of 

230 option numbers that are not technically NoCacheKey but handled by the 

231 application using this method). 

232 

233 >>> from aiocoap.numbers import GET 

234 >>> m1 = Message(code=GET) 

235 >>> m2 = Message(code=GET) 

236 >>> m1.opt.uri_path = ('s', '1') 

237 >>> m2.opt.uri_path = ('s', '1') 

238 >>> m1.opt.size1 = 10 # the only no-cache-key option in the base spec 

239 >>> m2.opt.size1 = 20 

240 >>> m1.get_cache_key() == m2.get_cache_key() 

241 True 

242 >>> m2.opt.etag = b'000' 

243 >>> m1.get_cache_key() == m2.get_cache_key() 

244 False 

245 >>> from aiocoap.numbers.optionnumbers import OptionNumber 

246 >>> ignore = [OptionNumber.ETAG] 

247 >>> m1.get_cache_key(ignore) == m2.get_cache_key(ignore) 

248 True 

249 """ 

250 

251 options = [] 

252 

253 for option in self.opt.option_list(): 

254 if option.number in ignore_options or (option.number.is_safetoforward() and option.number.is_nocachekey()): 

255 continue 

256 options.append((option.number, option.value)) 

257 

258 return (self.code, tuple(options)) 

259 

260 # 

261 # splitting and merging messages into and from message blocks 

262 # 

263 

264 def _extract_block(self, number, size_exp, max_bert_size): 

265 """Extract block from current message.""" 

266 if size_exp == 7: 

267 start = number * 1024 

268 size = max_bert_size 

269 else: 

270 size = 2 ** (size_exp + 4) 

271 start = number * size 

272 

273 if start >= len(self.payload): 

274 raise error.BadRequest("Block request out of bounds") 

275 

276 end = start + size if start + size < len(self.payload) else len(self.payload) 

277 more = True if end < len(self.payload) else False 

278 

279 payload = self.payload[start:end] 

280 blockopt = (number, more, size_exp) 

281 

282 if self.code.is_request(): 

283 return self.copy( 

284 payload=payload, 

285 mid=None, 

286 block1=blockopt 

287 ) 

288 else: 

289 return self.copy( 

290 payload=payload, 

291 mid=None, 

292 block2=blockopt 

293 ) 

294 

295 def _append_request_block(self, next_block): 

296 """Modify message by appending another block""" 

297 if not self.code.is_request(): 

298 raise ValueError("_append_request_block only works on requests.") 

299 

300 block1 = next_block.opt.block1 

301 if block1.more: 

302 if len(next_block.payload) == block1.size: 

303 pass 

304 elif block1.size_exponent == 7 and \ 

305 len(next_block.payload) % block1.size == 0: 

306 pass 

307 else: 

308 raise error.BadRequest("Payload size does not match Block1") 

309 if block1.start == len(self.payload): 

310 self.payload += next_block.payload 

311 self.opt.block1 = block1 

312 self.token = next_block.token 

313 self.mid = next_block.mid 

314 else: 

315 # possible extension point: allow messages with "gaps"; then 

316 # ValueError would only be raised when trying to overwrite an 

317 # existing part; it is doubtful though that the blockwise 

318 # specification even condones such behavior. 

319 raise ValueError() 

320 

321 def _append_response_block(self, next_block): 

322 """Append next block to current response message. 

323 Used when assembling incoming blockwise responses.""" 

324 if not self.code.is_response(): 

325 raise ValueError("_append_response_block only works on responses.") 

326 

327 block2 = next_block.opt.block2 

328 if not block2.is_valid_for_payload_size(len(next_block.payload)): 

329 raise error.UnexpectedBlock2("Payload size does not match Block2") 

330 if block2.start != len(self.payload): 

331 # Does not need to be implemented as long as the requesting code 

332 # sequentially clocks out data 

333 raise error.NotImplemented() 

334 

335 if next_block.opt.etag != self.opt.etag: 

336 raise error.ResourceChanged() 

337 

338 self.payload += next_block.payload 

339 self.opt.block2 = block2 

340 self.token = next_block.token 

341 self.mid = next_block.mid 

342 

343 def _generate_next_block2_request(self, response): 

344 """Generate a sub-request for next response block. 

345 

346 This method is used by client after receiving blockwise response from 

347 server with "more" flag set.""" 

348 

349 # Note: response here is the assembled response, but (due to 

350 # _append_response_block's workings) it carries the Block2 option of 

351 # the last received block. 

352 

353 next_after_received = len(response.payload) // response.opt.block2.size 

354 blockopt = optiontypes.BlockOption.BlockwiseTuple( 

355 next_after_received, False, response.opt.block2.size_exponent) 

356 

357 # has been checked in assembly, just making sure 

358 assert blockopt.start == len(response.payload) 

359 

360 blockopt = blockopt.reduced_to(response.remote.maximum_block_size_exp) 

361 

362 return self.copy( 

363 payload=b"", 

364 mid=None, 

365 token=None, 

366 block2=blockopt, 

367 block1=None, 

368 observe=None 

369 ) 

370 

371 def _generate_next_block1_response(self): 

372 """Generate a response to acknowledge incoming request block. 

373 

374 This method is used by server after receiving blockwise request from 

375 client with "more" flag set.""" 

376 response = Message(code=CHANGED, token=self.token) 

377 response.remote = self.remote 

378 if self.opt.block1.block_number == 0 and self.opt.block1.size_exponent > DEFAULT_BLOCK_SIZE_EXP: 

379 new_size_exponent = DEFAULT_BLOCK_SIZE_EXP 

380 response.opt.block1 = (0, True, new_size_exponent) 

381 else: 

382 response.opt.block1 = (self.opt.block1.block_number, True, self.opt.block1.size_exponent) 

383 return response 

384 

385 # 

386 # the message in the context of network and addresses 

387 # 

388 

389 

390 def get_request_uri(self, *, local_is_server=False): 

391 """The absolute URI this message belongs to. 

392 

393 For requests, this is composed from the options (falling back to the 

394 remote). For responses, this is largely taken from the original request 

395 message (so far, that could have been trackecd by the requesting 

396 application as well), but -- in case of a multicast request -- with the 

397 host replaced by the responder's endpoint details. 

398 

399 This implements Section 6.5 of RFC7252. 

400 

401 By default, these values are only valid on the client. To determine a 

402 message's request URI on the server, set the local_is_server argument 

403 to True. Note that determining the request URI on the server is brittle 

404 when behind a reverse proxy, may not be possible on all platforms, and 

405 can only be applied to a request message in a renderer (for the 

406 response message created by the renderer will only be populated when it 

407 gets transmitted; simple manual copying of the request's remote to the 

408 response will not magically make this work, for in the very case where 

409 the request and response's URIs differ, that would not catch the 

410 difference and still report the multicast address, while the actual 

411 sending address will only be populated by the operating system later). 

412 """ 

413 

414 # maybe this function does not belong exactly *here*, but it belongs to 

415 # the results of .request(message), which is currently a message itself. 

416 

417 if self.code.is_response(): 

418 refmsg = self.request 

419 

420 if refmsg.remote.is_multicast: 

421 if local_is_server: 

422 multicast_netloc_override = self.remote.hostinfo_local 

423 else: 

424 multicast_netloc_override = self.remote.hostinfo 

425 else: 

426 multicast_netloc_override = None 

427 else: 

428 refmsg = self 

429 multicast_netloc_override = None 

430 

431 proxyuri = refmsg.opt.proxy_uri 

432 if proxyuri is not None: 

433 return proxyuri 

434 

435 scheme = refmsg.opt.proxy_scheme or refmsg.remote.scheme 

436 query = refmsg.opt.uri_query or () 

437 path = refmsg.opt.uri_path 

438 

439 if multicast_netloc_override is not None: 

440 netloc = multicast_netloc_override 

441 else: 

442 if local_is_server: 

443 netloc = refmsg.remote.hostinfo_local 

444 else: 

445 netloc = refmsg.remote.hostinfo 

446 

447 if refmsg.opt.uri_host is not None or \ 

448 refmsg.opt.uri_port is not None: 

449 

450 host, port = hostportsplit(netloc) 

451 

452 host = refmsg.opt.uri_host or host 

453 port = refmsg.opt.uri_port or port 

454 

455 # FIXME: This sounds like it should be part of 

456 # hpostportjoin/-split 

457 escaped_host = quote_nonascii(host) 

458 

459 # FIXME: "If host is not valid reg-name / IP-literal / IPv4address, 

460 # fail" 

461 

462 netloc = hostportjoin(escaped_host, port) 

463 

464 # FIXME this should follow coap section 6.5 more closely 

465 query = "&".join(_quote_for_query(q) for q in query) 

466 path = ''.join("/" + _quote_for_path(p) for p in path) or '/' 

467 

468 fragment = None 

469 params = "" # are they not there at all? 

470 

471 # Eases debugging, for when thy raise from urunparse you won't know 

472 # which it was 

473 assert scheme is not None 

474 assert netloc is not None 

475 return urllib.parse.urlunparse((scheme, netloc, path, params, query, fragment)) 

476 

477 def set_request_uri(self, uri, *, set_uri_host=True): 

478 """Parse a given URI into the uri_* fields of the options. 

479 

480 The remote does not get set automatically; instead, the remote data is 

481 stored in the uri_host and uri_port options. That is because name resolution 

482 is coupled with network specifics the protocol will know better by the 

483 time the message is sent. Whatever sends the message, be it the 

484 protocol itself, a proxy wrapper or an alternative transport, will know 

485 how to handle the information correctly. 

486 

487 When ``set_uri_host=False`` is passed, the host/port is stored in the 

488 ``unresolved_remote`` message property instead of the uri_host option; 

489 as a result, the unresolved host name is not sent on the wire, which 

490 breaks virtual hosts but makes message sizes smaller. 

491 

492 This implements Section 6.4 of RFC7252. 

493 """ 

494 

495 parsed = urllib.parse.urlparse(uri) 

496 

497 if parsed.fragment: 

498 raise ValueError("Fragment identifiers can not be set on a request URI") 

499 

500 if parsed.scheme not in coap_schemes: 

501 self.opt.proxy_uri = uri 

502 return 

503 

504 if parsed.username or parsed.password: 

505 raise ValueError("User name and password not supported.") 

506 

507 if parsed.path not in ('', '/'): 

508 self.opt.uri_path = [urllib.parse.unquote(x) for x in parsed.path.split('/')[1:]] 

509 else: 

510 self.opt.uri_path = [] 

511 if parsed.query: 

512 self.opt.uri_query = [urllib.parse.unquote(x) for x in parsed.query.split('&')] 

513 else: 

514 self.opt.uri_query = [] 

515 

516 self.remote = UndecidedRemote(parsed.scheme, parsed.netloc) 

517 

518 is_ip_literal = parsed.netloc.startswith('[') or ( 

519 parsed.hostname.count('.') == 3 and 

520 all(c in '0123456789.' for c in parsed.hostname) and 

521 all(int(x) <= 255 for x in parsed.hostname.split('.'))) 

522 

523 if set_uri_host and not is_ip_literal: 

524 self.opt.uri_host = urllib.parse.unquote(parsed.hostname).translate(_ascii_lowercase) 

525 

526 # Deprecated accessors to moved functionality 

527 

528 @property 

529 def unresolved_remote(self): 

530 return self.remote.hostinfo 

531 

532 @unresolved_remote.setter 

533 def unresolved_remote(self, value): 

534 # should get a big fat deprecation warning 

535 if value is None: 

536 self.remote = UndecidedRemote('coap', None) 

537 else: 

538 self.remote = UndecidedRemote('coap', value) 

539 

540 @property 

541 def requested_scheme(self): 

542 if self.code.is_request(): 

543 return self.remote.scheme 

544 else: 

545 return self.request.requested_scheme 

546 

547 @requested_scheme.setter 

548 def requested_scheme(self, value): 

549 self.remote = UndecidedRemote(value, self.remote.hostinfo) 

550 

551 @property 

552 def requested_proxy_uri(self): 

553 return self.request.opt.proxy_uri 

554 

555 @property 

556 def requested_hostinfo(self): 

557 return self.request.opt.uri_host or self.request.unresolved_remote 

558 

559 @property 

560 def requested_path(self): 

561 return self.request.opt.uri_path 

562 

563 @property 

564 def requested_query(self): 

565 return self.request.opt.uri_query 

566 

567class UndecidedRemote( 

568 namedtuple("_UndecidedRemote", ("scheme", "hostinfo")), 

569 interfaces.EndpointAddress 

570 ): 

571 """Remote that is set on messages that have not been sent through any any 

572 transport. 

573 

574 It describes scheme, hostname and port that were set in 

575 :meth:`.set_request_uri()` or when setting a URI per Message constructor. 

576 

577 * :attr:`scheme`: The scheme string 

578 * :attr:`hostinfo`: The authority component of the URI, as it would occur 

579 in the URI. 

580 """ 

581 

582_ascii_lowercase = str.maketrans(string.ascii_uppercase, string.ascii_lowercase) 

583 

584_quote_for_path = quote_factory(unreserved + sub_delims + ':@') 

585_quote_for_query = quote_factory(unreserved + "".join(c for c in sub_delims if c != '&') + ':@/?') 

586 

587#: Result that can be returned from a render method instead of a Message when 

588#: due to defaults (eg. multicast link-format queries) or explicit 

589#: configuration (eg. the No-Response option), no response should be sent at 

590#: all. Note that per RFC7967 section 2, an ACK is still sent to a CON 

591#: request. 

592#: 

593#: Depercated; set the no_response option on a regular response instead (see 

594#: :meth:`.interfaces.Resource.render` for details). 

595NoResponse = Sentinel("NoResponse")