Optimizes procedures for posting the 100 MiB article.

But we're not yet ready to send such an article back to a client.  The
reason is most likely the fact that we read the entire article into
memory in fetch-article.  The new sensible idea is to not read the
body of articles into memory, but delay the body reading until the
last moment---of sending it out to the client.

Nevertheless, we're getting needlessly ambitious here because we don't
intend to support file attachments anyway.  We might do it just as an
exercise, if the end result looks good.  I think it's a good exercise
and I think we should do it.
This commit is contained in:
Circling Skies 2026-07-15 22:34:28 -03:00
parent 6a9fcc1ffc
commit 5a176692db

34
loop.nw
View file

@ -903,6 +903,21 @@ time. %% TODO
<<How to parse articles.lisp>>=
(defun parse-article (v)
(let ((pos (search #(13 10 13 10) v)))
(if pos
(make-article
:headers (map 'string #'code-char (subseq v 0 pos))
;; Create a zero-copy window into the original byte array.
:body (make-array (- (length v) (+ pos 4))
:element-type (array-element-type v)
:displaced-to v
:displaced-index-offset (+ pos 4)))
;; What should we do if no body is found? Is that legal NNTP?
(make-article
:headers (map 'string #'code-char v)
:body (make-array 0 :element-type (array-element-type v))))))
(defun parse-article-old (v)
(let ((parts (split-vector (vector 13 10 13 10) v nil :limit 2)))
(make-article :headers (map 'string #'code-char (car parts)) :body (cadr parts))))
@ -1784,10 +1799,27 @@ must have \verb|message-id|, \verb|subject|, \verb|from|,
@
Sometimes we parse an article and sometimes we want to undo that
parsing. Am I doing something wrong? I wonder.
parsing. Am I doing something wrong? I wonder. We surely are, but
now is not the time for fixing this.
<<Command post.lisp>>=
(defun unparse-article (parsed)
;; 1. Build the new headers as a standard string
(let* ((headers-str (with-output-to-string (s)
(dolist (h (parse-headers (article-headers parsed)))
(format s "~a: ~a~c~c" (str:capitalize (car h)) (cdr h) #\return #\linefeed))
;; Add the final \r\n separating headers from the body
(format s "~c~c" #\return #\linefeed)))
;; 2. Convert the string to a small byte array (~1 KB)
(headers-bytes (make-array (length headers-str)
:element-type '(unsigned-byte 8)
:initial-contents (map 'list #'char-code headers-str)))
;; 3. Get the displaced, zero-copy body array
(body-bytes (article-body parsed)))
;; 4. Let SBCL compile this into a fast C-level memcpy
(concatenate '(vector (unsigned-byte 8)) headers-bytes body-bytes)))
(defun unparse-article-old (parsed)
(data
(let ((ls))
(dolist (h (parse-headers (article-headers parsed)))