Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
django-tables2: hyperlink in cells
I have a view named events displaying two columns. One eventType and a second eventDates, listing dates. It looks like this: eventType eventDate foo 1900-01-01, 2010-02-02 bar 2000-01-01, 2010-02-02, 2010-03-03 The code for the view and the table looks like: def events(request, pk): data = some_module.get_data(pk) data_table = MyTable(data) return render(request, "app/events.html", {"table": data_table}) Where data is a list of dictionaries and MyTable is a class inheriting from django_tables2.Table: class MyTable(tables.Table): eventType = tables.Column() eventDates = tables.Column() A cell in the column eventDate can be either empty or containing several dates. It depends on pk. I can display that table. Now, I would like each date to be an hyperlink redirecting to another view. This view would be like that: def event_details(request, eventtype=et, eventdate=ed): data_event = some_module.get_event data(eventtype, eventdate) return render(request, "app/events_details.html", {"data_event ": data_event }) And here are my issues. First, I have not been able to have a dynamic number of hyperlink, varying from cell to cell. Second, the url should include the eventType and its date. E.g. taking the exemple above, cliking on 2010-02-02 should redirect to either: eventtype=foo_eventdate=2010-02-02 or eventtype=bar_eventdate=2010-02-02, depending on which 2020-02-02 I clik on. I have first tried to have several links in … -
Making an AI security app using django and react-native, but the output that is sent by django does not show on android and ios but does show on Web
Making a security app that will using a classification model detect who is at my door (members of my family for now) and show it on an App that i am making in react-native. Ive got most of the code figured out but the problem i am now facing is that when django sends the feed (image by image im presuming) the mobile apps dont show the feed (using expo go) but the web app does. Views.py: from django.shortcuts import render from django.views.decorators import gzip from django.http import StreamingHttpResponse import cv2 import threading import numpy as np from tensorflow.keras.models import load_model # Load the face recognition model model = load_model('./final_face_recognition_model.keras') # Define class labels class_labels = ["person1", "person2", "person3"] # Load OpenCV's pre-trained Haar Cascade for face detection face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml') # Define a function to preprocess frames for the model def preprocess_frame(frame, x, y, w, h): """ Extract the face region, resize, normalize, and reshape it for the model. """ face = frame[y:y+h, x:x+w] input_size = (160, 160) # Model's input size resized_face = cv2.resize(face, input_size) normalized_face = resized_face / 255.0 # Normalize pixel values reshaped_face = np.expand_dims(normalized_face, axis=0) # Add batch dimension return reshaped_face class VideoCamera(object): … -
no domain link when ı launched my website on aws elasticbeanstalk
ı get some errors on aws elastic beanstalk and ı can not get domain link, my website contains data but ı didnt use database when ı do configurations but errors are not about database? also ı get this error November 19, 2024 11:34:18 (UTC+3) INFO Environment health has transitioned from Pending to No Data. Initialization in progress (running for 16 minutes). There are no instances``` -
State is lost following Apple login via python social auth (AuthStateMissing)
I've added apple-id to a django project that already has facebook & twitter login implemented & working. From the application you can get to apple, login and then you come back to the /player/social/complete/{backend}/ path. Here there's an AuthStateMissing: Session value state missing exception. Sessions are using signed_cookies, SESSION_COOKIE_SAMESITE is currently "Lax" but I've tried to set that to None with no change. Besides the client/team/key settings I have also defined the following for Apple ID SOCIAL_AUTH_APPLE_ID_SCOPE = ["email", "name"] SOCIAL_AUTH_APPLE_ID_EMAIL_AS_USERNAME = ( True # If you want to use email as username ) I've tried various bits that I've found suggested on here and github issues, like specifying the state field for session storage. But nothing seems to resolve this issue and I'm not familiar enough with how this library works - I'd hoped it'd "just work" like apple usually suggest! -
Apple signin/login with django and flutter problem - invalid_client response
I'm having an issue with implementing Apple Sign-In/Login in my Django + Flutter application. Here's the flow I'm using: On the Flutter side, I'm using the following code to get credentials from Apple: SignInWithAppleButton( onPressed: () async { final credential = await SignInWithApple.getAppleIDCredential( scopes: [ AppleIDAuthorizationScopes.email, AppleIDAuthorizationScopes.fullName, ], ); print(credential); // Now send the credential (especially `credential.authorizationCode`) to your server to create a session // after they have been validated with Apple }, ); This returns information like identity_token and authorization_code. For test purposes, I send the authorization_code in a POST request to the endpoint https://appleid.apple.com/auth/token with the following details: Headers: { "Content-type": "application/x-www-form-urlencoded" } Body: { "client_id": "com.example.myapp", "client_secret": "example_client_secret", "grant_type": "authorization_code", "code": "my_example_authorization_code" } For generating the client_secret, I use the following Python script: # Load the private key with open(PRIVATE_KEY_PATH, "rb") as key_file: private_key = serialization.load_pem_private_key( key_file.read(), password=None, backend=default_backend() ) header = { "alg": "ES256", "kid": KEY_ID, } payload = { "iss": TEAM_ID, "iat": int(time.time()), "exp": int(time.time()) + 15777000, "aud": "https://appleid.apple.com", "sub": APP_ID, } client_secret = jwt.encode( payload, private_key, algorithm="ES256", headers=header ) print(client_secret) After all that steps, every time I send the POST request to Apple's /auth/token endpoint, I get the following response: {"error": "invalid_client"} Is there … -
get_absolute_url django admin session id missing
I'm using Django's "View on site" feature to navigate from the Django admin to a specific page in my application for a model instance. On this page, a GET request is made to retrieve data using a Django ModelViewSet. The issue is that even though I'm logged into the Django admin, when I visit the application page via "View on site," it says I'm "unauthenticated." Upon inspecting the request headers, I noticed that the session ID is not being sent. Here’s how I’ve implemented the get_absolute_url method: return ( reverse("website:emi_calculator") + f"?snowflake={self.snowflake}" ) Any help or guidance would be greatly appreciated! -
django docker poetry mysql driver problem
I'vse used django a milion times and docker half a million times but this time i can't get it to work Image: FROM python:3.11.5-slim-bookworm AS python-base Dockerfile: apt-get install --no-install-recommends -y python3-dev default-libmysqlclient-dev build-essential pkg-config Poetry mysqlclient = "^2.2.6" Django settings: 'ENGINE': 'django.db.backends.mysql' django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module. Did you install mysqlclient? Yes ... i did ... There is a lot of threads regarding this, i tried many variations but solutions from 2013 don't seem to work any more ... -
How to create a Django Admin link with preselected objects and a pre-filled action?
I am trying to generate a link that takes users to the Django Admin page for a specific model, with certain objects already selected and an action pre-filled in the dropdown menu. Here’s what I’ve tried so far: def get_admin_action_link(record, action): app_label = record._meta.app_label model = record._meta.model_name id = record.id env = f"{settings.env}.myurl.com" if settings.env else "http://localhost:8000" return f"{env}/admin/{app_label}/{model}?action={action}&_selected_action={id}" The generated link looks like this: http://localhost:8000/admin/app/mymodel?action=process&_selected_action=591 However, when I click on the link, it only takes me to the changelist view of the model in the admin. The objects aren’t selected, and the action isn’t pre-filled. -
How to Run Functions in Parallel or in the Background in Django Without Using Celery/Redis?
I have a use case in my Django application where I want to run two functions in parallel or execute a process in the background. I don't want to use third-party tools like Celery, Redis, or similar services for this. What I want to achieve: Safely execute tasks in parallel or in the background. Avoid slowing down the main request-response cycle. I've heard that making internal API calls via HTTP (e.g., calling Django endpoints from within the same project) can be a faster alternative. Is this true and safe? I don't want this to effected by the GIL in python -
Django API with Redis Cache Not Updating Immediately After POST/PUT/DELETE Requests
I'm developing a Django REST API using Django, Redis, and SQLite. The goal is to cache data for GET requests to improve performance, using Redis as the caching layer. Issue: When I create, update, or delete Category or Product instances, the changes are reflected in the database (as seen on the Django admin page), but not immediately in subsequent GET requests. The GET responses only reflect the changes after I restart the server. However, creating a new category with the same name is prevented, indicating the API can still read existing data from the database. app/views.py from django.shortcuts import render from django.core.cache import cache from rest_framework import viewsets, status from rest_framework.response import Response from rest_framework.decorators import action from .models import Category, Product from .serializers import CategorySerializer, ProductSerializer from django.conf import settings CACHE_TTL = getattr(settings, 'CACHE_TTL', 5) class CategoryViewSet(viewsets.ModelViewSet): queryset = Category.objects.all() serializer_class = CategorySerializer def list(self, request, *args, **kwargs): cache_key = 'store:categories' data = cache.get(cache_key) if not data: # Serialize the queryset to JSON before caching data = self.get_serializer(self.queryset, many=True).data cache.set(cache_key, data, CACHE_TTL) print('Cache set') print('Cache retrieved') return Response(data) def create(self, request, *args, **kwargs): cache.delete('store:categories') print('cache deleted') return super().create(request, *args, **kwargs) def update(self, request, *args, **kwargs): cache.delete('store:categories') return super().update(request, *args, … -
Caching daily data in Django, and caching more on request
I'm currently building a website where I each day want yesterday's financial data cached for the full day, to be quickly available to users. I also want to cache older data on request, e.g. if a user looks at data for some date in 2023, I also want to cache that, although for a shorter time (say maybe 1 h). Being new to Django, I've tried to read up on the different caching solutions that Django offers, but I'm a bit confused as to which would server me well. Does Django offer a solution fit for this purpose? Would it be easier to set up a manual caching process, which fetches cached data if it exists, and otherwise retrieves and caches it for a specfied time period? If so, what would such a solution look like? Thank you. -
Django all-auth extended SignupForm is valid but save method is not called
I am using Django all-auth in conjunction with django. I am also extending their default signup form. The form renders correctly, fields are validated correctly, the form returns as valid if checked but the save method isn't even called. These are my settings, When the submit button is pressed, the form posts fine. But then no user is created and through some testing the save method on the form just isn't called. forms.py class CustomSignupForm(stupidform): agreed_to_TC = forms.BooleanField(required=True) agreed_to_PA = forms.BooleanField(required=False) agreed_to_updates = forms.BooleanField(required=False) email = forms.EmailField(required=True) password = forms.CharField(label='Password confirmation', widget=forms.PasswordInput) password2 = forms.CharField(label='Password confirmation', widget=forms.PasswordInput) def clean_password(self): password = self.cleaned_data["password"] password2 = self.cleaned_data.get("password2") if password and password2 and password != password2: raise ValidationError("Passwords don't match") return password def save(self, request): user = super(CustomSignupForm, self).save(request) user.email = self.cleaned_data["email"] user.agreed_to_ads = self.cleaned_data["agreed_to_PA"] user.agreed_to_TC = self.cleaned_data["agreed_to_TC"] user.save() return user settings.py AUTH_USER_MODEL = 'dashboard.UserProf' ACCOUNT_USER_MODEL_USERNAME_FIELD = None ACCOUNT_EMAIL_REQUIRED = True ACCOUNT_USERNAME_REQUIRED = False ACCOUNT_AUTHENTICATION_METHOD = 'email' SECURE_SSL_REDIRECT = False ACCOUNT_DEFAULT_HTTP_PROTOCOL = "https" ACCOUNT_EMAIL_REQUIRED = True # ACCOUNT_EMAIL_VERIFICATION = "mandatory" ACCOUNT_FORMS = { 'signup': 'wishfor.forms.CustomSignupForm', } urls path('accounts/', include('allauth.urls')), -
WebSocket Connection Failure and Unexpected Socket Closure in Django Channels
I'm experiencing issues with establishing and maintaining a WebSocket connection in my Django project using Django Channels. I've set up a notification system that uses WebSockets to broadcast messages to connected clients. However, I'm encountering two errors: "WebSocket connection to 'ws://127.0.0.1:8000/ws/notification/broadcast/' failed" "Chat socket closed unexpectedly" Here's my code: routing.py: from django.urls import re_path from . import consumers websocket_urlpatterns = [ re_path(r'ws/notification/(?P<room_name>\w+)/$', consumers.NotificationConsumer.as_asgi()), ] views.py: def notification_view(request): return render(request,'person/notification.html',{'room_name': "broadcast"}) settings.py: CHANNEL_LAYERS = { 'default': { 'BACKEND': 'channels_redis.core.RedisChannelLayer', 'CONFIG': { "hosts": [('127.0.0.1', 6379)], }, }, } consumer_context_processor.py: from notification_app.models import BroadcastNotification def notifications(request): allnotifications = BroadcastNotification.objects.all() return {'notifications': allnotifications} models.py: class BroadcastNotification(models.Model): message = models.TextField() notification_image=models.ImageField(upload_to="notification",default="notification.jpg",blank=True,null=True) notification_link = models.URLField(max_length=10000, help_text="Add a valid URL",blank=True,null=True) broadcast_on = models.DateTimeField() sent = models.BooleanField(default=False) class Meta: ordering = ['-broadcast_on'] @receiver(post_save, sender=BroadcastNotification) def notification_handler(sender, instance, created, **kwargs): # call group_send function directly to send notificatoions or you can create a dynamic task in celery beat if created: schedule, created = CrontabSchedule.objects.get_or_create(hour = instance.broadcast_on.hour, minute = instance.broadcast_on.minute, day_of_month = instance.broadcast_on.day, month_of_year = instance.broadcast_on.month) task = PeriodicTask.objects.create(crontab=schedule, name="broadcast-notification-"+str(instance.id), task="notifications_app.tasks.broadcast_notification", args=json.dumps((instance.id,))) task.py: @shared_task(bind = True) def broadcast_notification(self, data): print(data) try: notification = BroadcastNotification.objects.filter(id = int(data)) if len(notification)>0: notification = notification.first() channel_layer = get_channel_layer() loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) loop.run_until_complete(channel_layer.group_send( "notification_broadcast", … -
Inline Serializer for Request Body Not Displaying in DRF Spectacular Documentation
I am working on a Django REST Framework (DRF) project using DRF Spectacular for API documentation. My ViewSet does not use a serializer because the logic does not require one, but I need to describe the request body shape in the API documentation. To achieve this, I am using DRF Spectacular's inline_serializer to define the request body schema dynamically. Here’s an example of how I’ve implemented it: from drf_spectacular.utils import extend_schema, inline_serializer from rest_framework import serializers from drf_spectacular.types import OpenApiTypes from rest_framework.response import Response class ExampleViewSet(viewsets.ViewSet): @extend_schema( request=inline_serializer( name="InlineFormSerializer", fields={ "str_field": serializers.CharField(), "int_field": serializers.IntegerField(), "file_field": serializers.FileField(), }, ), responses={200: OpenApiTypes.OBJECT}, # Placeholder for actual response schema ) def create(self, request, *args, **kwargs): # Business logic here return Response({"message": "Success"}) While the schema generation works without errors, the request body schema described via inline_serializer does not appear in the frontend API documentation (e.g., Swagger UI). The rest of the API is documented correctly. Steps I've Taken Confirmed that DRF Spectacular is installed and properly configured in settings.py REST_FRAMEWORK = { 'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema', } Verified that the schema generation command runs without issues Confirmed that inline_serializer is being used as per the documentation. Ensured the swagger-ui is accessible and displays other endpoints … -
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 61: invalid continuation byte
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 61: invalid continuation byte помогите решить проблему я хочу сделать миграцию бд в проекте в джанго с sqlite на postgres и при команде python manage.py возникает такая ошибка в чем проблема и как ее решить больше важнее как ее решить +- разобрался в чем конкретно проблема а вот как решить нигде не могу найти помогитее все перепробовал -
Django unit test with parallel db accesses giving database locked error
I am writing some Django unit tests to validate the behavior of my application when multiple clients access it at the same time. I have done this by writing tests like this: from django.test import TestCase from django.db import connection from django.db import transaction class ApplicationTestCase(TestCase): def test_parallel(self): @transaction.atomic def local_provision(i): with connection.cursor() as cursor: # Some SQL commands that read and write the person table. # These commands take less than one second. return f"done with {i}" with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: futures = [executor.submit(local_provision, i) for i in range(5)] for future in concurrent.futures.as_completed(futures): print(future.result()) When I run this unit test I get an immediate error: django.db.utils.OperationalError: database table is locked: person I tried changing the timeout in the settings.py file but that did not make any difference. Why isn't SQLite waiting until the file is unlocked and then retrying? How can I get SQLite to retry? -
having problem with managing general content on the website
I’m working on a full-stack project with a custom admin panel (built using Django and DRF) and a Next.js frontend. I’ve implemented CRUD functionalities for specific resources like users, products, and categories, tickets etc. Now I’m stuck on how to handle general website content like banners, announcements, and texts. Right now, I’m using static predefined divs on the frontend (e.g., a "main-page-banner" div), and I link these to specific records in the database (records have a key relevant to the pre-defined div and other fields to store data). But this approach has issues: Admins can’t add new sections or blocks dynamically I need a preview mode where admins can see draft changes before publishing them. What’s the best way to make my admin panel work like a CMS to manage general content (while keeping it simple)? Should I build everything from scratch, or is there a better approach? Thanks in advance! What’s the best way to make my admin panel work like a CMS to manage general content (while keeping it simple)? Should I build everything from scratch, or is there a better approach? Thanks in advance! -
How to properly integrate Opentelemetry while using fork process model for multiprocessing in python django app
We have a python/django application, in our app we are connecting to snowflake data warehouse and we have long running requests for reporting (sql queries are being used to fetch data from snowflake). To handle time consuming queries(and achieve multi processing) we were using Threading (Code is attached below). And We also had telemetry configured and it was working fine. Opentelemetry works fine while using Threads. Now We have shifted from Thread approach to fork process model for multi processing and telemetry broke down, its not working as expected. We are unable to get it work. We are using background_process_job decorator function to create a Thread on windows and Process on Linux, , When we test our code on windows(Thread is created), Telemetry works fine BUT When we test the same code on linux, a process is created instead of thread and telemetry does not work. Are we missing something? Why telemetry @tracer.start_as_current_span("send-message-core-thread") does not work while using fork process and works fine when using Threads? def background_process_job(function): def decorator(*args, **kwargs): # If windows, start a thread. if platform.system() == 'Windows': thread = Thread(target=function, args=args, kwargs=kwargs) thread.daemon = True thread.start() logger.debug(f"Thread started with id {thread.native_id}") else: # if linux, start … -
Disable Response Buffering on nginx for StreamingHTTPResponse
I have a django application running on AWS Elastic Beanstalk. Part of this application streams an OpenAI chat completion response to the user in chunks. When running with gunicorn locally, the response is streamed in chunks as expected. After deploying to AWS Elastic Beanstalk with a nginx server, the response is not streamed as it is in dev. It buffers until the entire response is ready and then returns it at once. I believe I need to disable response buffering from nginx. Below is my current structure. def generate_stream(): ai_response = StringIO() try: response = client.chat.completions.create( model="gpt-4o", messages=context_messages, temperature=0.7, max_tokens=4096, stream=True ) for chunk in response: content = chunk.choices[0].delta.content if content: ai_response.write(content) yield content try: ai_response_obj = Message.objects.create(conversation=conversation, content=ai_response.getvalue(), role="assistant") print(f"AI response saved: {ai_response_obj}") except Exception as e: print(f"Failed to save AI response: {e}") except (openai.APIConnectionError, openai.OpenAIError) as e: print(f"API connection error: {e}") yield ("ERROR: There was an issue processing your request. Please check your internet connection and try again.") except Exception as e: print(f"Unexpected error: {e}") yield "Error: An unexpected error occurred." response_server = StreamingHttpResponse(generate_stream(), content_type='text/plain') response_server['Cache-Control'] = 'no-cache' # prevent client cache response_server["X-Accel-Buffering"] = "no" response_server.status_code = status.HTTP_201_CREATED return response_server I also have a config file located in … -
Getting Access Denied Error When Hitting API (Although, API is working fine when running on LocalHost via Django)
I used GPT4all to create a RAG Pipeline with the help of langchain. When I use API for other purposes, it works fine but when I Hit the RAG pipeline, It gives access denied error. Error getting: Permission Error Root Cause: Unable to create model download directory in system profile. Access denied to 'C:\Windows\system32\config\systemprofile.cache' Environment:- Python Version: 3.12 Operating System: Windows Key Packages: * langchain * gpt4all * pydantic Full Error: {"error":"Error in Database Chain: Failed to create model download directory","traceback":"Traceback (most recent call last):\n File "C:\Python312\Lib\site-packages\gpt4all\gpt4all.py", line 323, in retrieve_model\n os.makedirs(DEFAULT_MODEL_DIRECTORY, exist_ok=True)\n File "", line 215, in makedirs\n File "", line 225, in makedirs\nPermissionError: [WinError 5] Access is denied: 'C:\\Windows\\system32\\config\\systemprofile\\.cache'\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "C:\inetpub\wwwroot\hashmove-ai\HMAI\Controllers\ConversationalAIv2_Controller.py", line 143, in post\n hmgpt_response = query_response(input_query, query_intent)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "C:\inetpub\wwwroot\hashmove-ai\HMAI_Business\ConversationalAIv2_Business.py", line 281, in query_response\n return response_functions.get(intent_response, irrelevant)(query)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "C:\inetpub\wwwroot\hashmove-ai\HMAI_Business\ConversationalAIv2_Business.py", line 251, in logistics\n chain = log_chain(load_llm(), vector_db(), memory(), log_prompt())\n ^^^^^^^^^^\n File "C:\inetpub\wwwroot\hashmove-ai\HMAI_Business\ConversationalAIv2_Business.py", line 90, in load_llm\n loaded_llm = GPT4All(\n ^^^^^^^^\n File "C:\Python312\Lib\site-packages\langchain_core\load\serializable.py", line 125, in init\n super().init(*args, **kwargs)\n File "C:\Python312\Lib\site-packages\pydantic\main.py", line 212, in init\n validated_self = self.pydantic_validator.validate_python(data, self_instance=self)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "C:\Python312\Lib\site-packages\pydantic\_internal\_decorators_v1.py", line 148, in _wrapper1\n return validator(values)\n ^^^^^^^^^^^^^^^^^\n File "C:\Python312\Lib\site-packages\langchain_core\utils\pydantic.py", line … -
Getting duplicate page inside when submitting a form
When I submit the form (i.e) placing a order I am getting a new duplicate page rendering inside my ship_now.html page . Tried changing logic in views which are creating form in a new html file using block content created new method for form submission instead of writing logic in same method. but it doesn't help . {% load static%} <html lang="en-GB"> Ship Now | TrackItNow <a href="/"> TrackItNow </div> </a> </div> <button type="button" class="jw-mobile-menu__button jw-mobile-toggle" aria-label="Toggle menu"> <span class="jw-icon-burger"></span> </button> </div> <header class="header-wrap js-topbar-content-container js-fixed-header-container"> <div class="header-wrap__inner"> <div class="header"> <div class="jw-header-logo"> <div class="jw-header jw-header-title-container jw-header-text jw-header-text-toggle" data-stylable="true"> <a id="jw-header-title" class="jw-header-title" href="/"> TrackItNow </a> </div> </div> </div> <nav class="menu jw-menu-copy"> <ul id="jw-menu" class="jw-menu jw-menu-horizontal"> <li class="jw-menu-item"> <a class="jw-menu-link" href="/track_home/" data-page-link-id="24182718"> <span class=""> Home </span> </a> </li> <li class="jw-menu-item jw-menu-is-active"> <a class="jw-menu-link js-active-menu-item" href="/ship_now/" data-page-link-id="24184780"> <span class=""> Ship Now </span> </a> </li> <li class="jw-menu-item"> <a class="jw-menu-link" href="/order_list/" data-page-link-id="24184796"> <span class=""> Track order </span> </a> </li> <li class="jw-menu-item"> <a class="jw-menu-link" href="/" data-page-link-id="24182760"> <span class=""> About Us </span> </a> </li> <li class="jw-menu-item"> <a class="jw-menu-link" href="/contact" data-page-link-id="24182761"> <span class=""> Contact </span> </a> </li> </ul> </nav> </div> </header> <div id="jw-slideshow" class="jw-slideshow jw-slideshow-toggle jw-slideshow--height-ratio jw-slideshow--parallax jw-slideshow--parallax-effect banner-sm" data-pause="7000" data-autoplay="1" data-transition="horizontal" data-ratio="0.33"> <div class="bx-wrapper"> <div class="bx-viewport"> <ul> … -
Performance issue (long serialization time) with Django rest framework serializers on large dataset like 1000 objects while fetching (read) the data
I am working on a Django application that uses Django Rest Framework to expose APIs. I am experiencing significant performance issues when serializing large data in Django Rest Framework. I have an endpoint that returns a list of objects e.g. 1000 records. The response is slow and taking around 5-6 seconds. This is not acceptable for the user experience. Environment Details: App Engine Runtime: Python 3.9, Django Version: 2.2.28, Django REST Framework Version: 3.12.4 For example, I have a Test model that has nearly 40 fields. When fetching a list of objects , the response time is about 5-6 seconds which is too slow for my use case. Here's the relevant code: Query: return self.ndb_class.all( namespace=self.get_namespace(), ancestor=ancestor_key ) Code snippet: queryset = self.filter_queryset(self.get_queryset()) page = self.paginate_queryset(queryset) if page is not None: self.get_related_data(page) serializer = self.get_serializer(page, many=True) return self.get_paginated_response(serializer.data) serializer = self.get_serializer(queryset, many=True) results = {'results': serializer.data} return Response(results) Profiling Toolbar shows that the query time isn't the issue, but the serialization step seems to be slow. And because of that, I have tested other serialization libraries, including those suggested in online references (such as DRF Serpy, Pydantic, Marshmallow, etc.), but I have not been able to reduce the serialization time … -
How to use Django BinaryField to store PNG images in postgres?
I need to store PNG files as blob/binary in the database and then be able to retrieve and show them. Here's my model that has a binary field to store the image: class ImageFile(models.Model): file = models.BinaryField(editable=True) I created a widget based on this answer: class BinaryFileInputWidget(forms.ClearableFileInput): def is_initial(self, value): return bool(value) def format_value(self, value): if self.is_initial(value): return f"{len(value)} bytes" def value_from_datadict(self, data, files, name): upload = super().value_from_datadict(data, files, name) if upload: return upload.read() And I used it in admin.py like this: @admin.register(ImageFile) class ImageFileAdmin(admin.ModelAdmin): list_display = ["id"] formfield_overrides = { models.BinaryField: {"widget": BinaryFileInputWidget()}, } Then I encode the file as base64 in the view: def image_view(request: HttpRequest, id: int): document_file = ImageFile.objects.filter(id=id).first() data = "" if document_file: data = base64.b64encode(document_file.file).decode(encoding="ascii") return render(request, "image.html", {"data": data}) <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Document</title> </head> <body> <pre>data:image/png;base64,{{ data }}</pre> <img src="data:image/png;base64,{{ data }}" alt="" /> </body> </html> There is no error and the data is shown in the pre element but the img element fails to load the image. I tried using some online converters to check if the data is valid but they show errors saying the base64 string is invalid. It loads … -
Django LiveServerTestCase live server shuts down prematurely
I've been trying to run some functional tests (LiveServerTestCase) with Selenium for a Django app. For demonstration it's a note taking app. When I'm trying to simulate note creation, it requires authentication, so in the setup it involves going into admin dashboard, inserting credentials into the form fields, then submitting. After that, I programmatically redirect to the page where I am supposed to write notes, but as soon as I click on submit, it induces a forbidden error. It says that the instance running the test is an AnonymousUser, ergo, the submission does not follow through. Do keep in mind that submission is in the end of the testcase, so I can't help but presuppose that the server ends prematurely. Is there a flag or configuration that I'm missing as to why this occurs? An example code of the testcase: class NoteTakingTest(StaticLiveServerTestCase): @classmethod def setUpClass(cls): super().setUpClass() chrome_options = Options() cls.selenium = WebDriver(options=chrome_options) cls.selenium.implicitly_wait(10) @classmethod def tearDownClass(cls): cls.selenium.quit() return super().tearDownClass() def setUp(self): super().setUp() user = User.objects.create_superuser(email="admin@testing.com", password="admin") self.selenium.get(f"{self.live_server_url}/admin/login") email = self.selenium.find_element(By.ID, "id_username") password = self.selenium.find_element(By.ID, "id_password") email.send_keys("admin@testing.com") password.send_keys("admin") form = self.selenium.find_element(By.ID, "login-form") form.submit() def access(self, url): self.selenium.get(self.live_server_url + url) def test_pipeline_page_access(self): pipeline_url = reverse("notes") self.access(pipeline_url) self.assertFalse(self.selenium.title.startswith("404")) wait = WebDriverWait(self.selenium, 10) submit_button … -
In what unit does django display response time?
I am using Django to serve an application, and I noticed some slowing down recently. So I went and checked the console that serves the server, that usually logs lines of this format : <date_time> "GET <path> HTTP/1.1" <HTTP_STATUS> <response_time> What I thought was the response time in milliseconds is apparently not, as I get values that would be ludicrous (example 3923437 for a query that when timed in python directly takes 0.936 seconds). I'm pretty sure it's a response time though, as it's always scaled with the time I wait. Can someone explain to me what that number is ? I couldn't find where this default log is documented.
