* [2024-06-28 Fri] ** well, i'm setting up sbcl and sly? because i imagine using Common Lisp for my attempt at building screen sharing for the XR glasses for the cool runtime access, right? ** well, just flake shell and (use-package sly) is enough for .lisp files to get stuff yes, i'm still using C-M-f and stuff to move around sexps but things are fine so, what's next? well, this is nice high level description dbus https://www.freedesktop.org/wiki/Software/dbus/ and here's something to start usage https://blog.macrolet.net/posts/DBus-and-PolicyKit-from-Common-Lisp.html from https://github.com/death/dbus and even lots of examples, but i think i will need a big book on Common Lisp to learn new things? * [2024-06-29 Sat] ** going through the dbus guide, sounds neat https://develop.kde.org/docs/features/d-bus/introduction_to_dbus/ that qdbus and qdbus-viewer are in #+begin_src bash $ nix shell nixpkgs#kdePackages.qttools #+end_src ** now let's read about calling methods * the receiving of the signals * all the different tabs i had ** overall freedesktop portal dbus api docs https://docs.flatpak.org/en/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.ScreenCast ** searching github for 'get-managed-objects usage https://github.com/search?q=get-managed-objects&type=code ** searchin github for 'define-singlan-handler usage examples https://github.com/search?q=dbus%3Adefine-dbus-signal-handler&type=code ** example in cl-notify of calling a method 'with-introspected-object https://github.com/Lautaro-Garcia/cl-notify/blob/main/src/dbus-protocol.lisp#L27 #+begin_src lisp (defmacro with-dbus-method-inovaction ((result-var method &rest args) &body body) (alexandria:with-gensyms (bus notifications-object) `(dbus:with-open-bus (,bus (dbus:session-server-addresses)) (dbus:with-introspected-object (,notifications-object ,bus "/org/freedesktop/Notifications" "org.freedesktop.Notifications") (let ((,result-var (,notifications-object "org.freedesktop.Notifications" ,method ,@args))) ,@body))))) #+end_src ** example in cl-notify of working with signals https://github.com/Lautaro-Garcia/cl-notify/blob/main/src/signals.lisp they register single handler, and add callbacks dynamically & call them from the registered function ** dbus library example for publishing objects https://github.com/death/dbus/blob/master/examples/publish.lisp ** my request for help with interactive async calls https://github.com/death/dbus/issues/31 ** old ticket on dbus repo about enabling introspection https://github.com/death/dbus/issues/23 ** example of signal-handler in stumpwm-thingy https://github.com/lokedhs/stumpwm-dbus/blob/9cf0b52876111e777da27a42e3c81269b97c0005/src/pidgin.lisp#L16 ** ref for symbold in dbus package https://quickref.common-lisp.net/dbus.html#The-dbus_002fconnections-system ** blog article with basics of usage https://blog.macrolet.net/posts/DBus-and-PolicyKit-from-Common-Lisp.html ** big article about iolib https://pages.cs.wisc.edu/~psilord/blog/data/iolib-tutorial/tutorial.html it's the one that's used to create multiplexer for communication i tried to figure out what it means to have event-loops ** docs on dbus Screenshots portal https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Screenshot.html the methods and stuff ** docs on dbus type system https://dbus.freedesktop.org/doc/dbus-specification.html#type-system i.e how to understand types of methods and properties #+begin_src lisp CL-USER> (dbus:sigexp "xsaysas") (:INT64 :STRING (:ARRAY :BYTE) :STRING (:ARRAY :STRING)) #+end_src function signature explanation can show what should lisp types be https://github.com/death/dbus/issues/17#issuecomment-418920440 ** example of flameshot how they get Screenshot via dbus https://github.com/flameshot-org/flameshot/blob/c1dac52231024174faa68a29577129ebca125dff/src/utils/screengrabber.cpp#L59 ** dbus methods for Notification https://specifications.freedesktop.org/notification-spec/notification-spec-latest.html#signals ** dbus methods of ScreenCast https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html how to start them etc ** dbus spec on Request & Response for the async methods https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Request.html#org-freedesktop-portal-request ** repo for xdg-portal for wayland https://github.com/emersion/xdg-desktop-portal-wlr ** the python snippet to check that ScreenCast via xdg-portal works https://gitlab.gnome.org/-/snippets/19 * [2024-07-20 Sat] ** and now i want to try to get display of stuff into gtk window, i guess ** doc about cl bindings to gtk3 https://www.crategus.com/books/cl-gtk/gtk-tutorial.html#example-window-simple-c ** well, i guess i could go and try to read guide on pipewire with C examples https://docs.pipewire.org/page_tutorial.html this is tutorial for the system and here is example program that maybe does similar thing https://docs.pipewire.org/video-play_8c-example.html * [2024-07-28 Sun] ** ok, now i'm trying to get frames from the pipewire 'out' node and one option i'm exploring is gstreamer which is a framework for creating applications that stream video / audio with some kind of ability to define \ declare pipeline and then start and then use api somehow there's c api, and gobject accessibility layer so either c to common lisp, or gobject common lisp thingy maybe exist ** reading docs https://gstreamer.freedesktop.org/documentation/frequently-asked-questions/using.html?gi-language=c added via gst_all_1.gstreamer ** more readin gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink this is test for how it's installed needed to add plugins https://nixos.wiki/wiki/GStreamer ** i guess i can try to go with the tutorials? https://gstreamer.freedesktop.org/documentation/tutorials/index.html?gi-language=c but with common lisp over gobject? i think i'm already using this in the gtk bindings playground? maybe not https://cl-gtk2.common-lisp.dev/doc/gobject/Creating-GObjects-classes-and-implementing-GInterfaces.html #+begin_quote GObject binding at the moment provides only limited scenarios of creating GObject classes. #+end_quote ** and there's something https://github.com/BohongHuang/cl-gobject-introspection-wrapper which is based on older https://github.com/andy128k/cl-gobject-introspection https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=gobject they are packaged in nix, this is just unbelievable how much is done by humanity, omg * [2024-08-15 Thu] ** getting back after a long while. so, i was at the creating a window, not knowing how to stream video from pipewire out node so. what to do, huh i think gtk3 bindings library was separate from gobject and glib but maybe it doesn't matter? or maybe i should figure out glib bindings and use gtk3 though it? ** well, there's something simple with opening a window [[file:gtk-playground.lisp::defun example-window-simple (]] ** and something extra simple with gstreamer opening separate window for test video source [[file:gstreamer/tutorial-hello-world.lisp]] ** maybe try to open gstreamer window with screen share? ok, when i try example of pipewire window on screenshare source i get error that pipewiresrc is not found and that's right, i used system-wide gst-launch-1.0 and when i tried with one in flake shell - it also didn't work so. tried removing all from shell env, then not found namespace Gst (defvar *gstreamer* (gir:require-namespace "Gst")) so. yeah. maybe i need to install dev headers or something? ** so. for moving the video around. chatgpt recommended something named 'videobox' like i can declare it in the pipeline and then get reference to it and use to set video coordinates, huh? https://gstreamer.freedesktop.org/documentation/videobox/index.html?gi-language=python the thing exists but i have zero idea whether it's accessible in a way that's recommended #+begin_src lisp (defun for-screencast () (gir:invoke (*gstreamer* 'init) '()) (let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch) "pipewiresrc path=49 ! videoconvert ! videobox top=0 left=0 right=0 bottom=0 ! videoconvert ! autovideosink" )) (videobox (gir:invoke (pipeline 'get-element-by-name) "videobox"))) ;; Set the initial state to playing (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing)) ;; Function to move the video (defun move-video (top left) (gir:invoke (videobox 'set-property) 'top top) (gir:invoke (videobox 'set-property) 'left left)) ;; Example: Move the video 10 pixels down and 20 pixels to the right (move-video 10 20))) #+end_src and similar advice from claude, huh #+begin_src lisp (defun for-screencast () (gir:invoke (*gstreamer* 'init) '()) (let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch) "pipewiresrc path=49 ! videoconvert ! videobox name=move ! videoscale ! autovideosink")) (move-element (gir:invoke (pipeline 'get-by-name) "move"))) (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing)) (values pipeline move-element))) (defun move-video (move-element left top) (gir:set-property move-element "left" left) (gir:set-property move-element "top" top)) ;; Usage: (multiple-value-bind (pipeline move-element) (for-screencast) ;; Move video 50 pixels to the right and 30 pixels down (move-video move-element 50 30) ;; Your main loop or other logic here ) #+end_src let's try this out * [2024-08-16 Fri] ** new day of the vacation the experiment with moving the video, right? could it be that both examples are fake? i don't seem to find both 'get-element things trying to find anything about dynamism in gstreamer https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines/ looks like this is about changing elements of the pipeline and it's possible that parameters of the pipeline elements (pads?) are also static ** options for transforming frames? *** videotransform : A combination of scaling, 90-degree-step rotation, https://github.com/Freescale/gstreamer-imx/blob/master/README.md horizontal/vertical flipping, and color space conversion operations. These elements are able to render video overlay compositions from GstVideoOverlayCompositionMeta data. They also implement the GstVideoDirection interface and have a "video-direction" property that handles rotation and flipping. Through this property, it is possible to configure these elements to auto-rotate images according to the information in image-orientation tags. *** glimagesink https://gstreamer.freedesktop.org/documentation/opengl/glimagesink.html?gi-language=python gst-launch-1.0 -v videotestsrc ! video/x-raw ! glimagesink has error ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Failed to create EGLDisplay from native display Additional debug info: ../ext/gl/gstglimagesink.c(1136): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink ERROR: pipeline doesn't want to preroll. so maybe opengl is not available? how to check? *** gltransformation https://gstreamer.freedesktop.org/documentation/opengl/gltransformation.html?gi-language=python here examples seem to be for images, not video and same error, let's figure out the opengl check ** somewhat categorized things? * [2024-08-21 Wed] ** visiting sister, returning back last time i got stuck on video displaying frozen when i tried to add appsink just re-checking my basic gstreamer pipeline and screencast start - thing works but i want to clean it up i guess, to allow for simpler retries? ** so, for the appsnk there are signals https://gstreamer.freedesktop.org/documentation/app/appsink.html?gi-language=python#appsink::new-sample new-sample #+begin_src python def new_sample_callback (appsink, udata): #python callback for the 'new-sample' signal #+end_src Signal that a new sample is available. but how do i connect to a callback from common lisp and gobject introspection https://github.com/andy128k/cl-gobject-introspection does the gitlab repo has info on callbacks? can't seem to call method 'pull-sample' maybe article will help? https://gstreamer.freedesktop.org/documentation/application-development/advanced/pipeline-manipulation.html?gi-language=python#grabbing-data-with-appsink well, this looks like method on gst namespace? "gst_app_sink_pull_sample()" https://gstreamer.freedesktop.org/documentation/applib/gstappsink.html?gi-language=python#gst_app_sink_pull_sample there's article for python bindings https://lifestyletransfer.com/how-to-use-gstreamer-appsink-in-python/ * [2024-08-22 Thu] ** hey, i think this is progress i am able to call appsink 'pull-sample after all (for some reason, i don't know what i was doing differently) new error: >>> got to process sample with # but in the code for processing of the sample some type errors Backtrace: 0: ((:METHOD GIR::THIS-OF (T)) T) [fast-method] 1: (GIR:PROPERTY T DATA) 2: (PROCESS-FRAME #) 3: ((LAMBDA (APPSINK) :IN START-PROCESSING) #) ok, but when i have struct instance, how do i introspect it? ** let's just go and read documentation for the type that's returned from pull-sample, sure also there's that python code, right? doc 1: get sample on appsink https://people.freedesktop.org/~tsaunier/documentation/gst-plugins-base-app-1.0/gstappsink.html?gi-language=c#gst_app_sink_pull_sample doc 2: GstSample class https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=c#GstSample so on GstSample everything is taken as results of funciton calls? let's try that well, maybe i don't need get-info on sample, it returns null, maybe i need it on buffer? doc 3: get-buffer on sample https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=python#gst_sample_get_buffer doc 4: buffer class info https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_peek_memory ** i'm recommended to look into GstBuffer.map https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_map it returns some https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstmemory.html?gi-language=python#GstMapInfo ** struggling, searching for answers https://github.com/andy128k/cl-gobject-introspection/issues/73#issuecomment-755316735 Examples of inspection of a repository can be found in test files. https://github.com/andy128k/cl-gobject-introspection/blob/master/test/test1.lisp and i mean it's possible that referencing members of a structure is not implemented? so, huh using internal functions #+begin_src lisp (data (gir:fields-dict-of (gir:struct-class-of map-info))) #+end_src i see >>>> and data has size NIL and buf ((memory . #) (flags . #) (data . #) (size . #) (maxsize . #) (user_data . #) (_gst_reserved . #)) ; Evaluation aborted on T CAPTURING-FRAMES> so. there are data and size fields.